I read this article today and thought LW might find it interesting. The key finding is that in a number of different experiments, simple "self-affirmations" (such as writing about relationships with your friends or something else that makes you feel good about yourself) make people more open to changing their mind in cases where changing their mind would be damaging to their self-image. The proposed explanation is that people need to maintain a certain level of self-worth, and one way they do that is by refusing to accept evidence that would damage their sense of self-worth. But if they have a high enough sense of self-worth, they are less likely to do this. I haven't reviewed any of these studies personally, but the idea makes some sense and sounds pretty easy to try. Hat tip to Dan Keys for putting me onto the idea. I searched LW for "Sherman self-affirmation" and didn't see this discussed anywhere on LW, but I didn't look very hard.

Title: Accepting Threatening Information: Self–Affirmation and the Reduction of Defensive Biases

Authors: David K. Sherman and Geoffrey L. Cohen

Citation details: Current Directions in Psychological Science August 2002 vol. 11 no. 4 119-123

Abstract: Why do people resist evidence that challenges the validity of long–held beliefs? And why do they persist in maladaptive behavior even when persuasive information or personal experience recommends change? We argue that such defensive tendencies are driven, in large part, by a fundamental motivation to protect the perceived worth and integrity of the self. Studies of social–political debate, health–risk assessment, and responses to team victory or defeat have shown that people respond to information in a less defensive and more open–minded manner when their self–worth is buttressed by an affirmation of an alternative source of identity. Self–affirmed individuals are more likely to accept information that they would otherwise view as threatening, and subsequently to change their beliefs and even their behavior in a desirable fashion. Defensive biases have an adaptive function for maintaining self–worth, but maladaptive consequences for promoting change and reducing social conflict.

Key quotes: "Pro-choice partisans and pro-life partisans were presented with a debate between two activists on opposite sides of the abortion dispute….However, this confirmation bias was sharply attenuated among participants who affirmed a valued source of self-worth (by writing about a personally important value, such as their relations with friends)....although all participants left the debate feeling more confident in their beliefs about abortion than they had before, this polarization in attitude was significantly reduced among self-affirmed participants (cf. Lord et al., 1979)."  p. 120

"In one study (Cohen et al., 2000), devout opponents and proponents of capital punishment were presented with a persuasive scientific report that contradicted their beliefs about the death penalty’s effectiveness as a deterrent for crime....the responses of participants who received an affirmation of a valued self-identity (by writing about a personally important value, or by being provided with positive feedback on an important skill) proved more favorable.Self affirmed participants were less critical of the reported research, they suspected less bias on the part of the authors, and they even changed their overall attitudes toward capital punishment in the direction of the report they read." p. 121

"In one study, athletes who had just completed an intramural volleyball game assessed the extent to which each of a series of factors contributed to their team’s victory or defeat. As in past research (Lau & Russell, 1980),winners made more internal attributions for their victories than losers did for their defeats. However, among athletes who had reflected on an important value irrelevant to athletics, this self-serving bias was attenuated." p. 122

In Common sense as a prior, I used the example of prayer as a practice that is probably adaptive but the people who adopt it may not know why it is adaptive. I wrote:

Another striking example is bedtime prayer. In many Christian traditions I am aware of, it is common to pray before going to sleep. And in the tradition I was raised in, the main components of prayer were listing things you were grateful for, asking for forgiveness for all the mistakes you made that day and thinking about what you would do to avoid similar mistakes in the future, and asking God for things. Christians might say the point of this is that it is a duty to God, that repentance is a requirement for entry to heaven, or that asking God for things makes God more likely to intervene and create miracles. However, I think these activities are reasonable for different reasons: gratitude journals are great, reflecting on mistakes is a great way to learn and overcome weaknesses, and it is a good idea to get clear about what you really want out of life in the short-term and the long-term.

…I think it would be better still to introduce a different routine that serves similar functions—this is something I have done in my own life…

Someone recently wrote to me asking about my routine. I wrote this person an answer, so I thought I might as well share it with others. I have a number of structured routines like this that I find helpful and have considered sharing more widely, so this post will also serve as a test for whether I should share these routines. (These routines include: planning the day and tracking your time, planning and evaluating a project, doing a literature search, keeping a record of personal principles, reading and evaluating a paper, weekly review, and a few others that are less developed.)

Below, I offer and explanation of my routine, a template for following it, and give examples of what it looks like when I have used it. I have been doing this for about 6 weeks and I spend 5-15 minutes doing this per day. I was raised in a very religious family, and did something pretty similar for about the first 18 years of my life. I think it is good, but I don’t think the effect size/my tracking ability combo allows me to confidently distinguish between “it’s a placebo” and “it actually works” on the basis of my personal experience. I do it because it intuitively makes sense to me, it fits with some practices that I think are likely to be adaptive, it seems good so far, it seems good from a common sense perspective, some impressive people I know do similar things, and I’ve been told that psychological research on gratitude journals supports the idea. (Also, I don't mind benefits from "mere" placebos.)

One quick point of caution is that I would be careful about framing this as “atheist prayers” in your head. I framed it that way for a while and thought it would be a good idea to do it, but “atheist prayers” just sounds silly. On the other hand, “daily reflection” just sounds reasonable. I found framing it this way made me substantially more motivated to actually follow the process.

A detailed explanation of my process

1.       Getting started

a.       Download the document “Daily reflection.”

2.       Step by step

a.       At the end of the work day or before going to sleep, open up “Daily reflection.”

b.      Copy and paste the template for today’s entry.

c.       Fill in today’s date, e.g. 18 August 2013.

d.      Under “What went well today/what am I grateful for?”

                                                               i.      List things you feel good about doing recently or things you enjoyed today.

                                                             ii.      List general things you have noticed lately and appreciate, even if they are not recent.

                                                            iii.      (This is supposed to help you notice good things in life and seek out more of the good things.)

e.      Under “Where would I like to improve? What principles could I follow in the future in order to improve?”

                                                               i.      List any mistakes you think you made today.

                                                             ii.      Try to think about principles you could follow to avoid similar mistakes in the future.

                                                            iii.      If any of the principles seem useful or generally applicable, save them in another document, titled e.g. “My Principles.” I review my principles roughly monthly, and get reminders when I add new ones. I took this idea from Ray Dalio.

                                                           iv.      (This is supposed to help you learn from mistakes and identify, manage, and/or overcome personal weaknesses.)

b.      Under “What do I hope for in the future?”

                                                               i.      List upcoming challenges and opportunities that you hope go well.

                                                             ii.      List long-term priorities that you hope go well, especially ones you haven’t thought about lately or have been neglecting.

                                                          iii.      (This is supposed to help you keep track of what you really want out of life in the short-term and the long-term.)

3.       Other notes      

a.       If it is inconvenient to make notes in this document and I’m not going to open up the document, I will sometimes do my daily reflection in my head.

b.      I generally list 2-5 things under each category per day.

c.       I find this complements well with tracking your time. If you do track your time, you can look at how you spent your day and think about what was productive and what was unproductive. This helps with identifying items for the first and second steps.

Daily entries (template)

1.       Date:

a.       What went well today/what am I grateful for?

b.      Where would I like to improve? What principles could I follow in the future in order to improve?

c.       What do I hope for in the future?

2.       Date:

a.       What went well today/what am I grateful for?

b.      Where would I like to improve? What principles could I follow in the future in order to improve?

c.       What do I hope for in the future?

3.       Date:

a.       What went well today/what am I grateful for?

b.      Where would I like to improve? What principles could I follow in the future in order to improve?

c.       What do I hope for in the future?

Examples of daily entries (stripped of anything personal or embarrassing)

1.       9 August 2013

a.       What went well today/what am I grateful for?

                                                               i.      Got my post drafted

                                                             ii.      Great talk with [friend]

                                                            iii.      Enjoyed dinner

                                                           iv.      Talking to [other friend] was useful

b.      Where would I like to improve? What principles could I follow in the future in order to improve?

                                                               i.      Spending too much on dinners

                                                             ii.      Spending too much time criticizing people with dumb views

c.       What do I hope for in the future?

                                                               i.      Evaluate my projects well after they are done

                                                             ii.      Use my framework for evaluating topics to work on

2.       13 August 2013

a.       What went well today/what am I grateful for?

                                                               i.      Really enjoyed the weekly review

                                                             ii.      Glad I e-mailed a number of people to engage them on their perspectives

                                                            iii.      Glad I came up with the idea that 80K say what it is going to change and be held accountable for making the changes

b.      Where would I like to improve? What principles could I follow in the future in order to improve?

                                                               i.      I spent too much time checking the LW blog in response to my stuff

                                                             ii.      I’m not sure how useful it was for me to be involved with this prioritization institute stuff

                                                            iii.      I didn’t do a good job filling out my time tracker

                                                           iv.      Maybe [friend] is right that I didn’t do a good job as I think defending my common sense prior post

c.       What do I hope for in the future?

                                                               i.      Get my GCRI talk outline done

                                                             ii.      Get to the point where I can do 10 pull-ups

                                                            iii.      Review my common sense prior project

                                                           iv.      Share my productivity procedures with others

This article has been cross-posted at http://effective-altruism.com/.

An existential risk is a risk “that threatens the premature extinction of Earth-originating intelligent life or the permanent and drastic destruction of its potential for desirable future development,” (Bostrom, 2013). Nick Bostrom has argued that

“[T]he loss in expected value resulting from an existential catastrophe is so enormous that the objective of reducing existential risks should be a dominant consideration whenever we act out of an impersonal concern for humankind as a whole. It may be useful to adopt the following rule of thumb for such impersonal moral action:

Maxipok: Maximize the probability of an “OK outcome,” where an OK outcome is any outcome that avoids existential catastrophe.”

There are a number of people in the effective altruism community who accept this view and cite Bostrom’s argument as their primary justification. Many of these people also believe that the best ways of minimizing existential risk involve making plans to prevent specific existential catastrophes from occurring, and believe that the best giving opportunities must be with charities that primarily focus on reducing existential risk. They also appeal to Bostrom’s argument to support their views. (Edited to add: Note that Bostrom himself sees maxipok as neutral on the question of whether the best methods of reducing existential risk are very broad and general, or highly targeted and specific.) For one example of this, see Luke Muehlhauser’s comment:

“Many humans living today value both current and future people enough that if existential catastrophe is plausible this century, then upon reflection (e.g. after counteracting their unconscious, default scope insensitivity) they would conclude that reducing the risk of existential catastrophe is the most valuable thing they can do — whether through direct work or by donating to support direct work.”

I now think these views require some significant adjustments and qualifications, and given these adjustments and qualifications, their practical implications become very uncertain. I still believe that what matters most about what we do is how our actions affect humanity’s long-term future potential, and I still believe that targeted existential risk reduction and research is a promising cause, but it now seems unclear whether targeted existential risk reduction is the best area to look for ways of making the distant future go as well as possible. It may be and it may not be, and which is right probably depends on many messy details about specific opportunities, as well as general methodological considerations which are, at this point, highly uncertain. Various considerations played a role in my reasoning about this, and I intend to talk about more of them in greater detail in the future. I’ll talk about just a couple of these considerations in this post.

In this post, I argue that:

1. Though Bostrom’s argument supports the conclusion that maximizing humanity’s long term potential is extremely important, it does not provide strong evidence that reducing existential risk is the best way of maximizing humanity’s future potential. There is a much broader class of actions which may affect humanity’s long-term potential, and Bostrom’s argument does not uniquely favor existential risk over other members in this class.
2. A version of Bostrom’s argument better supports a more general view: what matters most is that we make path-dependent aspects of the far future go as well as possible. There are important questions about whether we should accept this more general view and what its practical significance is, but this more general view seems to be a strict improvement on the view that minimizing existential risk is what matters most.
3. The above points favor very broad, general, and indirect approaches to shaping the far future for the better, rather than thinking about very specific risks and responses, though there are many relevant considerations and the issue is far from settled.

I think some prominent advocates of existential risk reduction already agree with these general points, and believe that other arguments, or other arguments together with Bostrom’s argument, establish that direct existential risk reduction is what matters most. This post is most relevant to people who currently think Bostrom’s arguments may settle the issues discussed above.

Path-dependence and trajectory changes

In thinking about how we might affect the far future, I've found it useful to use the concept of the world's development trajectory, or just trajectory for short. The world's development trajectory, as I use the term, is a rough summary way the future will unfold over time. The summary includes various facts about the world that matter from a macro perspective, such as how rich people are, what technologies are available, how happy people are, how developed our science and culture is along various dimensions, and how well things are going all-things-considered at different points of time. It may help to think of the trajectory as a collection of graphs, where each graph in the collection has time on the x-axis and one of these other variables on the y-axis.

With that concept in place, consider three different types of benefits from doing good. First, doing something good might have proximate benefits—this is the name I give to the fairly short-run, fairly predictable benefits that we ordinarily think about when we cure some child's blindness, save a life, or help an old lady cross the street. Second, there are benefits from speeding up development. In many cases, ripple effects from good ordinary actions speed up development. For example, saving some child's life might cause his country's economy to develop very slightly more quickly, or make certain technological or cultural innovations arrive more quickly. Third, our actions may slightly or significantly alter the world's development trajectory. I call these shifts trajectory changes. If we ever prevent an existential catastrophe, that would be an extreme example of a trajectory change. There may also be smaller trajectory changes. For example, if some species of dolphins that we really loved were destroyed, that would be a much smaller trajectory change.

The concept of a trajectory change is closely related to the concept of path dependence in the social sciences, though when I talk about trajectory changes I am interested in effects that persist much longer than standard examples of path dependence. A classic example of path dependence is our use of QWERTY keyboards. Our keyboards could have been arranged in any number of other possible ways. A large part of the explanation of why we use QWERTY keyboards is that it happened to be convenient for making typewriters, that a lot of people learned to use these keyboards, and there are advantages to having most people use the same kind of keyboard. In essence, there is path dependence whenever some aspect of the future could easily have been way X, but it is arranged in way Y due to something that happened in the past, and now it would be hard or impossible to switch to way X. Path dependence is especially interesting when way X would have been better than way Y. Some political scientists have argued that path dependence is very common in politics. For example, in an influential paper (with over 3000 citations) Pierson (2000, p. 251) argues that:

Specific patterns of timing and sequence matter; a wide range of social outcomes may be possible; large consequences may result from relatively small or contingent events; particular courses of action, once introduced, can be almost impossible to reverse; and consequently, political development is punctuated by critical moments or junctures that shape the basic contours of social life.

The concept of a trajectory change is also closely related to the concept of a historical contingency. If Thomas Edison had not invented the light bulb, someone else would have done it later. In this sense, it is not historically contingent that we have light bulbs, and the most obvious benefits from Thomas Edison inventing the light bulb are proximate benefits and benefits from speeding up development. Something analogous is probably true of many other technological innovations such as computers, candles, wheelbarrows, object-oriented programming, and the printing press. Some important examples of historical contingencies: the rise of Christianity, the creation of the US Constitution, and the writings of Karl Marx. Various aspects of Christian morality influence the world today in significant ways, but the fact that those aspects of morality, in exactly those ways, were part of a dominant world religion was historically contingent. And therefore events like Jesus's death and Paul writing his epistles are examples of trajectory changes. Likewise, the US Constitution was the product of deliberation among a specific set of men, the document affects government policy today and will affect it for the foreseeable future, but it could easily have been a different document. And now that the document exists in its specific legal and historical context, it is challenging to make changes to it, so the change is somewhat self-reinforcing.

Some small trajectory changes could be suboptimal

Persistent trajectory changes that do not involve existential catastrophes could have great significance for shaping the far future. It is unlikely that the far future will inherit many of our institutions exactly as they are, but various aspects of the far future—including social norms, values, political systems, and perhaps even some technologies—may be path dependent on what happens now, and sometimes in suboptimal ways. In general, it is reasonable to assume that if there is some problem that might exist in the future and we can do something to fix it now, future people would also be able to solve that problem. But if values or social norms change, they might not agree that some things we think are problems really are problems. Or, if people make the wrong decisions now, certain standards or conventions may get entrenched, and resulting problems may be too expensive to be worth fixing. For further categories of examples of path-dependent aspects of the far future, see these posts by Robin Hanson.

The astronomical waste argument and trajectory changes

Bostrom’s argument only works if reducing existential risk is the most effective way of maximizing humanity’s future potential. But there is no robust argument that trying to reduce existential risk is a more effective way of shaping the far future than trying to create other positive trajectory changes. Bostrom’s argument for the overwhelming importance of reducing existential risk can be summarized as follows:

1. The expected size of humanity's future influence is astronomically great.
2. If the expected size of humanity's future influence is astronomically great, then the expected value of the future is astronomically great.
3. If the expected value of the future is astronomically great, then what matters most is that we maximize humanity’s long-term potential.
4. Some of our actions are expected to reduce existential risk in not-ridiculously-small ways.
5. If what matters most is that we maximize humanity’s future potential and some of our actions are expected to reduce existential risk in not-ridiculously-small ways, what it is best to do is primarily determined by how our actions are expected to reduce existential risk.
6. Therefore, what it is best to do is primarily determined by how our actions are expected to reduce existential risk.

Call that the “astronomical waste” argument.

It is unclear whether premise (5) is true because it is unclear whether trying to reduce existential risk is the most effective way of maximizing humanity’s future potential. For all we know, it could be more effective to try to create other positive trajectory changes. Clearly, it would be better to prevent extinction than to improve our social norms in a way that indirectly makes the future go one millionth better, but, in general, “X is a bigger problem than Y” is only a weak argument that “trying to address X is more important than trying to address Y.” To be strong, the argument must be supplemented by looking at many other considerations related to X and Y, such as how much effort is going into solving X and Y, how tractable X and Y are, how much X and Y could use additional resources, and whether there are subsets of X or Y that are especially strong in terms of these considerations.

Bostrom does have arguments that speeding up development and providing proximate benefits are not as important, in themselves, as reducing existential risk. And these arguments, I believe, have some plausibility. Since we don’t have an argument that reducing existential risk is better than trying to create other positive trajectory changes and an existential catastrophe is one type of trajectory change, it seems more reasonable for defenders of the astronomical waste argument to focus on trajectory changes in general. It would be better to replace the last two steps of the above argument with:

4’   Some of our actions are expected to change our development trajectory in not-ridiculously-small ways.

5’.  If what matters most is that we maximize humanity’s future potential and some of our actions are expected to change our development trajectory in not-ridiculously-small ways, what it is best to do is primarily determined by how our actions are expected to change our development trajectory.

6’.  Therefore, what it is best to do is primarily determined by how our actions are expected to change our development trajectory.

This seems to be a strictly more plausible claim than the original one, though it is less focused.

In response to the arguments in this post, which I e-mailed him in advance, Bostrom wrote a reply (see the end of the post). The key comment, from my perspective, is:

“Many trajectory changes are already encompassed within the notion of an existential catastrophe.  Becoming permanently locked into some radically suboptimal state is an xrisk.  The notion is more useful to the extent that likely scenarios fall relatively sharply into two distinct categories---very good ones and very bad ones.  To the extent that there is a wide range of scenarios that are roughly equally plausible and that vary continuously in the degree to which the trajectory is good, the existential risk concept will be a less useful tool for thinking about our choices.  One would then have to resort to a more complicated calculation.  However, extinction is quite dichotomous, and there is also a thought that many sufficiently good future civilizations would over time asymptote to the optimal track.”

I agree that a key question here is whether there is a very large range of plausible equilibria for advanced civilizations, or whether civilizations that manage to survive long enough naturally converge on something close to the best possible outcome. The more confidence one has in the second possibility, the more interesting existential risk is as a concept. The less confidence one has in the second possibility, the more interesting trajectory changes in general are. However, I would emphasize that unless we can be highly confident in the second possibility, it seems that we cannot be confident that reducing existential risk is more important than creating other positive trajectory changes because of the astronomical waste argument alone. This would turn on further considerations of the sort I described above.

Broad and narrow strategies for shaping the far future

Both the astronomical waste argument and the fixed up version of that argument conclude that what matters most is how our actions affect the far future. I am very sympathetic to this viewpoint, abstractly considered, but I think its practical implications are highly uncertain. There is a spectrum of strategies for shaping the far future that ranges from the very targeted (e.g., stop that asteroid from hitting the Earth) to very broad (e.g., create economic growth, help the poor, provide education programs for talented youth), with options like “tell powerful people about the importance of shaping the far future” in between. The limiting case of breadth might be just optimizing for proximate benefits or for speeding up development. Defenders of the astronomical waste argument tend to be on the highly targeted end of this spectrum. I think it’s a very interesting question where on this spectrum we should prefer to be, other things being equal, and it’s a topic I plan to return to in the future.

The arguments I’ve offered above favor broader strategies for shaping the far future, though they don’t settle the issue. The main reason I say this is that the best ways of creating positive trajectory changes may be very broad and general, whereas the best ways of reducing existential risk may be more narrow and specific. For example, it may be reasonable to try to assess, in detail, questions like, “What are the largest specific existential risks?” and, “What are the most effective ways of reducing those specific risks?” In contrast, it seems less promising to try to make specific guesses about how we might create smaller positive trajectory changes because there are so many possibilities and many trajectory changes do not have significance that is predictable in advance. No one could have predicted the persistent ripple effects that Jesus's life had, for example. In other cases—such as the framing of the US Constitution—it's clear that a decision has trajectory change potential, but it would be hard to specify, in advance, which concrete measures should be taken. In general, it seems that the worse you are at predicting some phenomenon that is critical to your plans, the less your plans should depend on specific predictions about that phenomenon. Because of this, promising ways to create positive trajectory changes in the world may be more broad than the most promising ways of trying to reduce existential risk specifically. Improving education, improving parenting, improving science, improving our political system, spreading humanitarian values, or otherwise improving our collective wisdom as stewards of the future could, I believe, create many small, unpredictable positive trajectory changes.

I do not mean to suggest that broad approaches are necessarily best, only that people interested in shaping the far future should take them more seriously than they currently do. The way I see the trade-off between highly targeted strategies and highly broad strategies is as follows. Highly targeted strategies for shaping the far future often depend on highly speculative plans, often with many steps, which are hard to execute. We often have very little sense of whether we are making valuable progress on AI risk research or geo-engineering research. On the other hand, highly broad strategies must rely on implicit assumptions about the ripple effects of doing good in more ordinary ways. It is very subtle and speculative to say how ordinary actions are related to positive trajectory changes, and estimating magnitudes seems extremely challenging. Considering these trade-offs in specific cases seems like a promising area for additional research.

Summary

In this post, I argued that:

1. The astronomical waste argument becomes strictly more plausible if we replace the idea of minimizing existential risk with the idea of creating positive trajectory changes.
2. There are many ways in which our actions could unpredictably affect our general development trajectory, and therefore many ways in which our actions could shape the far future for the better. This is one reason to favor broad strategies for shaping the far future.

The trajectory change perspective may have other strategic implications for people who are concerned about maximizing humanity’s long-term potential. I plan to write about these implications in the future.[i]

Comment from Nick Bostrom on this post

[What follows is an e-mail response from Nick Bostrom. He suggested that I share his comment along with the post. Note that I added a couple of small clarifications to this post (noted above) in response to Bostrom's comment.]

One can arrive at a more probably correct principle by weakening, eventually arriving at something like 'do what is best' or 'maximize expected good'.  There the well-trained analytic philosopher could rest, having achieved perfect sterility.  Of course, to get something fruitful, one has to look at the world not just at our concepts.

Many trajectory changes are already encompassed within the notion of an existential catastrophe.  Becoming permanently locked into some radically suboptimal state is an xrisk.  The notion is more useful to the extent that likely scenarios fall relatively sharply into two distinct categories---very good ones and very bad ones.  To the extent that there is a wide range of scenarios that are roughly equally plausible and that vary continuously in the degree to which the trajectory is good, the existential risk concept will be a less useful tool for thinking about our choices.  One would then have to resort to a more complicated calculation.  However, extinction is quite dichotomous, and there is also a thought that many sufficiently good future civilizations would over time asymptote to the optimal track.

In a more extended and careful analysis there are good reasons to consider second-order effects that are not captured by the simple concept of existential risk.  Reducing the probability of negative-value outcomes is obviously important, and some parameters such as global values and coordination may admit of more-or-less continuous variation in a certain class of scenarios and might affect the value of the long-term outcome in correspondingly continuous ways.  (The degree to which these complications loom large also depends on some unsettled issues in axiology; so in an all-things-considered assessment, the proper handling of normative uncertainty becomes important.  In fact, creating a future civilization that can be entrusted to resolve normative uncertainty well wherever an epistemic resolution is possible, and to find widely acceptable and mutually beneficial compromises to the extent such resolution is not possible---this seems to me like a promising convergence point for action.)

It is not part of the xrisk concept or the maxipok principle that we ought to adopt some maximally direct and concrete method of reducing existential risk (such as asteroid defense): whether one best reduces xrisk through direct or indirect means is an altogether separate question.