In four paragraphs I’m going to claim, “It is highly likely reading this article will increase your chance of living forever”. I’m pretty sure you won’t disagree with me. First, however, I’d like to talk about how much I don’t like Monopoly.

I play a lot of Monopoly, because I am forced into it – against my will – by friends, family, work-related-bonding etc. I understand this is a controversial opinion, but I really, really don’t like Monopoly – there is very little scope for creative play. In fact there is so little scope for create play I spotted that I could win at Monopoly, in a probabilistic sense, by going online and looking up the optimal allocation of houses to properties and valuation of houses in the ‘bargaining’ mid-game. For a while, the fact that nobody but me played ‘perfect’ Monopoly meant I won nearly every game, and I felt much better about playing because games tended to conclude more quickly when one player was a soulless, utility-hungry robot – it left me more time to concentrate on the stuff I actually enjoyed, which was socialising.

But Monopoly, despite being an almost completely deterministic dice-rolling game, hides unexpected complexity; a salutary lesson for an aspiring rationalist. Winning the game was completely secondary to my actual aim, which was forcing the game to take as little time as possible. I realised a few months ago that it didn’t matter who won, as long as somebody won quickly, and it was very unlikely the strategy optimised for one player was the same as the strategy optimised for all of them. As a consequence, I reran the computer simulations I built and developed an optimal ‘turn reducing’ strategy (it won’t surprise you to know that the basic rule is ‘play with as much variance as you possibly can’; having one maverick player lowers the average number of turns to the first bankruptcy, and bankruptcy is gamebreaking in Monopoly).

I agree that I could lower the number of turns even more by simply flipping over the board and storming out when someone suggests I play, but let’s assume I am also trying to balance a nebulously-defined but nonetheless real value of ‘not losing all my friends’, which is satisfied when I play a risky-but-exciting strategy and not satisfied when I constantly demand to play games I find fun. The point is, I had what Kuhn calls a ‘paradigm shift’ – once I realised that my goal when playing Monopoly was not to win as quickly as possible, but to ensure anyone wins as quickly as possible I was able to greatly, greatly increase my utility with no troublesome side-effects.

I’m relating this story to you because noticing my aims and strategy weren’t perfectly aligned improved my experience of Monopoly without doing anything difficult like hacking my motivation, and I’m sure you have similar stories of paradigm shifts improving your experience of a certain event (I hear people talk about the day they discovered coding was fun once they learned the rules, or maths was awesome once they got past the spadework. That has yet to happen to me, but my experience at thrashing my friends at a children’s board games means I can totally relate). What’s striking with these paradigm shifts is how obvious the conclusion seems in retrospect, and how opaque it seemed before the lightbulb moment. With that in mind, let me make a claim you might find concerning; “The aims and strategy of people who want to live forever are highly likely to be out of alignment”. In particular, from what I read on LW and other pro-cryo communities, the strategy-space explored is vastly smaller than the strategy-space of all possible cryonics strategies. Indeed, the strategy space explored by people who want to live forever is – in some ways – smaller than that explored by me while trying to get out of playing tedious boardgames. I’m going to talk about that strategy space a little in this article, mostly with the aim of triggering a ‘lightbulb moment’ – if there are any to be had – in readers a lot more committed to cryonics than me. To draw an obvious conclusion, if there are such lightbulb moments to be had, it is highly likely reading this article will increase your chance of living forever by increasing the size of the cryonics strategy space you consider.

That the strategy space explored is small is pretty hard to disagree with; there is an option to freeze or not-freeze in the first place, go with Alcor or The Cryonics Institute (or possibly KryoRuss), go for your full body or just your head and – maybe – whether to hang on for plastination or begin investing in cryonics insurance now. As far as I can tell, more ‘fringe’ options are not discussed with very much regularity. A search of the LW archives turned up this thread which was along similar lines, but didn’t trigger anything like the discussion I thought it would; this surprises me – when the ‘prize’ for picking a marginal improvement in your cryonics strategy that doubles your chance of revivification is that you double your chance of living forever, I’m highly surprised the cryonics strategy space is not exhaustively searched at this point, certainly amongst people who turn rationality into an art form.

For example, there are at least three ways I can think of to raise your chance of being successfully frozen:

·         (Sensible) Redundancy cryonics: Make redundant copies of the information you intend to preserve. For example, MRI scans of your brain and detailed notes on your reactions to certain stimuli. In the event that current technology almost-but-not-quite preserves information in the brain, your notes and images might help future scientists reconstruct your personality. You might even go further and send hippocampal slices to multiple cryonics facilities, gambling on the fact that the increased probability of at least one facility’s survival outweighs the lower probability of revivification from a single hippocampal slice.

·         (Sensible) Diversified cryonics: In addition to cryonics, employ some other strategy(s) which might result in you living forever but which are as completely uncorrelated with the success or failure of cryonics as you can manage given that ‘the complete destruction of the earth on a molecular level by a malevolent alien race’ correlates with many bad outcomes and few good ones. I actually have a list of about ten of these, which I will happily make available on request (i.e. I’ll write another discussion post about them if people are interested) but I don’t want the whole discussion of this post to be about this one single issue, which it was when I tried the content of the post out on my friend. This is about the cryonics strategy-space only, not the living-forever strategy space, which is much bigger.

·         (Inadvisable) Suicide cryonics: Calculate the point at which your belief in the utility of cryonics outweighs the expected utility of the rest of your life (this will likely come a few seconds before the average age of death in your demographic). Kill yourself in the most cryonics-friendly way you can imagine, which I suspect will involve injecting yourself with toxic cryoprotectants on top of a platform suspended over a large vat of liquid nitrogen so that when you collapse, you collapse into the nitrogen and freeze yourself (which should limit the amount of time the dead brain is at body-temperature). If you are not concerned about your body, you should also try to decapitate yourself as you fall to raise the surface area to volume ratio of the object you are trying to freeze.

Here are three ways that raise your chance of successfully remaining frozen:

·         (Sensible) Positive cryonics: Lobby for laws that ensure the government protects your body. Either lobby for these laws directly (I talked about a ‘right to not-death’ in my last post on this subject) or promise to report to future!USA’s equivalent of the Department of Defence to see if they can weaponise any microbes on you after you’re unfrozen. Remember that we’re talking in terms of expected utility here; the chance that such lobbying is effective is minute, but it might be an effective way to spend your twilight years if you would otherwise be unproductive.

·         (Sensible, but worryingly immoral) Negative cryonics: Sabotage as many cryonics labs as possible before going under, or lobby for laws that make it illegal to freeze yourself which only come into force after you die. This raises the chances that you are the James Bedford of modern cryonics and society has a particular interest in keeping your body safe. Note that though sabotaging an entire lab is difficult and illegal, trashing the field of cryonics itself is pretty easy and socially high-status because people already think it’s pretty weird – you’d predict that at least some detractors of cryonics are actually extremely pro-cryonics and trying to raise their chances of being kept frozen as a cultural curiosity rather than as only one of millions of corpsicles.

·         (Sensible if your name is Lex Luthor, otherwise implausible) Ninja cryonics: Build a cryonics pod yourself, with enough liquid nitrogen to keep you frozen for several thousand years, known only to the highly trusted individual who transfers your cryo-preserved body from Alcor to this location (if you could somehow get yourself into an unprotected far-earth orbit after freezing this would be perfect). Hope that your pod is discovered by friendly future-humans before you run out of coolant. This is insurance against the possibility that society destroys all cryonics labs somehow and then later regrets it (although, now I think about it, someone following this strategy certainly wouldn’t tell anyone about it on a public forum…)

Here are three ways that raise your chance of successfully being revived:

·         (Sensible if legal) Compound-interest cryonics: Devote a small chunk of your resources towards a fund which you expect to grow faster than the rate of inflation, with exponential growth (the simplest example would be a bank account with a variable rate that pays epsilon percent higher than the rate of inflation in perpetuity). Sign a contract saying the person(s) who revive you receive the entire pot. Since after a few thousand years the pot will nominally contain almost all the money in the world this strategy will eventually incentivise almost the entire world to dedicate itself to seeking your revival. Although this strategy will not work if postscarcity happens before unfreezing, it collapses into the conventional cryonics problem and therefore costs you no more than the opportunity cost of spending the capital in the fund before you die. (Although apparently this is illegal)

·         (Sensible) Cultural-value cryonics: Freeze yourself with something which is relatively cheap now, but you predict might be worth a lot of money in the future. I suspect that – for example – rare earth metals or gold might be a decent guess at something that will increase in value whatever society does, but the real treasure trove will be things like first-editions of books you expect might become classics in the future, original paintings by artists who might become very trendy in the 25^th Century or photographs of an important historic event which will become disputed or lionised in the future (my best bet would be anything involving the relationship between China and America if we’re talking a few centuries, and pre-technology parts of Africa if we’re talking millennia). It’s hard to believe even a post-singularity society won’t have some social signalling remaining, so you’ve got a respectable chance of finding a buyer for these artefacts. These fantastically valuable artefacts will be used to pay your way in a society where – thanks to the Flynn effect – you will have an IQ which breaks the curve at the ‘dangerously stupid’ end and you might not be able to survive otherwise. Be careful nobody knows you’re doing this, otherwise your cryopod will be raided like an Egyptian tomb! Even disregarding this financial advice, it might be a good idea to ensure you freeze yourself with e.g. a beloved pet, or the complete works of Shakespeare. This ensures that even if future society is so totally different to what you were expecting you will still have some information-age artefacts to protect you from culture-shock.

·         (Inadvisable and high-risk) Game-theory cryonics: Set up an alarm on your cryonics pod that unthaws you after five hundred years. This is insurance against the possibility that society is able to unfreeze you, but chooses not to, since no society would just let you die (you hope). You could go more supervillain-y than this by planting a deadly bomb somewhere, timed to go off in five hundred years unless you enter a 128-digit disarming key. This should incentivise society to develop revivification processes as a matter of urgency. Bear in mind if it is easier for future society to develop extremely strong counter-cryptography or radiation shielding your plan may backfire as research that would have been undertaken in cryopreservation is redeployed to stop your diabolical scheme.

I think most of these strategies have never been written about before, and of those that have been written about they have all been throwaway thought experiments on LW. Given that the strategy space of cryonics strategies is much bigger than cryonics advocates appear to instinctively gravitate around, I conclude that it is very unlikely there has been a serious effort to optimise the cryonics process beyond the scientific advances made by Alcor (and hence it is very unlikely we have all hit upon the optimal strategy by chance). This is especially true because the optimal strategy in some cases depends on the probability that the future resembles certain kinds of predictions, and I know people disagree over those predictions on LW. For example, the ratio of culturally-valuable artefacts to sanity-preserving artefacts you should take with you probably depends on the relative likelihood you assign that a post-scarcity or post-singularity world will be the one to revive you. I’m not in a very good position to make that particular judgement myself, but I am in a good position to say that there is a very real opportunity cost to considering a narrow strategy space when considering life-extending strategies, just as there is an opportunity cost when considering over-narrow Monopoly strategies. In the first case, the impact of your decision might result in you throwing your life away. In the second, it only feels like it does.

Previously: Contrarian Excuses, The Correct Contrarian Cluster, What is bunk?, Common Sense as a Prior, Trusting Expert Consensus, Prefer Contrarian Questions.

Robin Hanson once wrote:

On average, contrarian views are less accurate than standard views. Honest contrarians should admit this, that neutral outsiders should assign most contrarian views a lower probability than standard views, though perhaps a high enough probability to warrant further investigation. Honest contrarians who expect reasonable outsiders to give their contrarian view more than normal credence should point to strong outside indicators that correlate enough with contrarians tending more to be right.

I tend to think through the issue in three stages:

1. When should I consider myself to be holding a contrarian^[1] view? What is the relevant expert community?
2. If I seem to hold a contrarian view, when do I have enough reason to think I’m correct?
3. If I seem to hold a correct contrarian view, what can I do to give other people good reasons to accept my view, or at least to take it seriously enough to examine it at length?

I don’t yet feel that I have “answers” to these questions, but in this post (and hopefully some future posts) I’d like to organize some of what has been said before,^[2] and push things a bit further along, in the hope that further discussion and inquiry will contribute toward significant progress in social epistemology.^[3] Basically, I hope to say a bunch of obvious things, in a relatively well-organized fashion, so that less obvious things can be said from there.^[4]

In this post, I’ll just address stage 1. Hopefully I’ll have time to revisit stages 2 and 3 in future posts.

Is my view contrarian?

World model differences vs. value differences

Is my effective altruism a contrarian view? It seems to be more of a contrarian value judgment than a contrarian world model,^[5] and by “contrarian view” I tend to mean “contrarian world model.” Some apparently contrarian views are probably actually contrarian values.

Expert consensus

Is my atheism a contrarian view? It’s definitely a world model, not a value judgment, and only 2% of people are atheists.

But what’s the relevant expert population, here? Suppose it’s “academics who specialize in the arguments and evidence concerning whether a god or gods exist.” If so, then the expert population is probably dominated by academic theologians and religious philosophers, and my atheism is a contrarian view.

We need some heuristics for evaluating the soundness of the academic consensus in different fields. ^[6]

For example, we should consider the selection effects operating on communities of experts. If someone doesn’t believe in God, they’re unlikely to spend their career studying arcane arguments for and against God’s existence. So most people who specialize in this topic are theists, but nearly all of them were theists before they knew the arguments.

Perhaps instead the relevant expert community is “scholars who study the fundamental nature of the universe” — maybe, philosophers and physicists? They’re mostly atheists. ^[7] This is starting to get pretty ad-hoc, but maybe that’s unavoidable.

What about my view that the overall long-term impact of AGI will be, most likely, extremely bad? A recent survey of the top 100 authors in artificial intelligence (by citation index)^[8] suggests that my view is somewhat out of sync with the views of those researchers.^[9] But is that the relevant expert population? My impression is that AI experts know a lot about contemporary AI methods, especially within their subfield, but usually haven’t thought much about, or read much about, long-term AI impacts.

Instead, perhaps I’d need to survey “AGI impact experts” to tell whether my view is contrarian. But who is that, exactly? There’s no standard credential.

Moreover, the most plausible candidates around today for “AGI impact experts” are — like the “experts” of many other fields — mere “scholastic experts,” in that they^[10] know a lot about the arguments and evidence typically brought to bear on questions of long-term AI outcomes.^[11] They generally are not experts in the sense of “Reliably superior performance on representative tasks” — they don’t have uniquely good track records on predicting long-term AI outcomes, for example. As far as I know, they don’t even have uniquely good track records on predicting short-term geopolitical or sci-tech outcomes — e.g. they aren’t among the “super forecasters” discovered in IARPA’s forecasting tournaments.

Furthermore, we might start to worry about selection effects, again. E.g. if we ask AGI experts when they think AGI will be built, they may be overly optimistic about the timeline: after all, if they didn’t think AGI was feasible soon, they probably wouldn’t be focusing their careers on it.

Perhaps we can salvage this approach for determining whether one has a contrarian view, but for now, let’s consider another proposal.

Mildly extrapolated elite opinion

Nick Beckstead instead suggests that, at least as a strong prior, one should believe what one thinks “a broad coalition of trustworthy people would believe if they were trying to have accurate views and they had access to [one’s own] evidence.”^[12] Below, I’ll propose a modification of Beckstead’s approach which aims to address the “Is my view contrarian?” question, and I’ll call it the “mildly extrapolated elite opinion” (MEEO) method for determining the relevant expert population. ^[13]

First: which people are “trustworthy”? With Beckstead, I favor “giving more weight to the opinions of people who can be shown to be trustworthy by clear indicators that many people would accept, rather than people that seem trustworthy to you personally.” (This guideline aims to avoid parochialism and self-serving cognitive biases.)

What are some “clear indicators that many people would accept”? Beckstead suggests:

IQ, business success, academic success, generally respected scientific or other intellectual achievements, wide acceptance as an intellectual authority by certain groups of people, or success in any area where there is intense competition and success is a function of ability to make accurate predictions and good decisions…

Of course, trustworthiness can also be domain-specific. Very often, elite common sense would recommend deferring to the opinions of experts (e.g., listening to what physicists say about physics, what biologists say about biology, and what doctors say about medicine). In other cases, elite common sense may give partial weight to what putative experts say without accepting it all (e.g. economics and psychology). In other cases, they may give less weight to what putative experts say (e.g. sociology and philosophy).

Hence MEEO outsources the challenge of evaluating academic consensus in different fields to the “generally trustworthy people.” But in doing so, it raises several new challenges. How do we determine which people are trustworthy? How do we “mildly extrapolate” their opinions? How do we weight those mildly extrapolated opinions in combination?

This approach might also be promising, or it might be even harder to use than the “expert consensus” method.

My approach

In practice, I tend to do something like this:

• To determine whether my view is contrarian, I ask whether there’s a fairly obvious, relatively trustworthy expert population on the issue. If there is, I try to figure out what their consensus on the matter is. If it’s different than my view, I conclude I have a contrarian view.
• If there isn’t an obvious trustworthy expert population on the issue from which to extract a consensus view, then I basically give up on step 1 (“Is my view contrarian?”) and just move to the model combination in step 2 (see below), retaining pretty large uncertainty about how contrarian my view might be.

When do I have good reason to think I’m correct?

Suppose I conclude I have a contrarian view, as I plausibly have about long-term AGI outcomes,^[14] and as I might have about the technological feasibility of preserving myself via cryonics.^[15] How much evidence do I need to conclude that my view is justified despite the informed disagreement of others?

I’ll try to tackle that question in a future post. Not surprisingly, my approach is a kind of model combination and adjustment.

1. I don’t have a concise definition for what counts as a “contrarian view.” In any case, I don’t think that searching for an exact definition of “contrarian view” is what matters. In an email conversation with me, Holden Karnofsky concurred, making the point this way: “I agree with you that the idea of ‘contrarianism’ is tricky to define. I think things get a bit easier when you start looking for patterns that should worry you rather than trying to Platonically define contrarianism… I find ‘Most smart people think I’m bonkers about X’ and ‘Most people who have studied X more than I have plus seem to generally think like I do think I’m wrong about X’ both worrying; I find ‘Most smart people think I’m wrong about X’ and ‘Most people who spend their lives studying X within a system that seems to be clearly dysfunctional and to have a bad track record think I’m bonkers about X’ to be less worrying.”  ↩

2. For a diverse set of perspectives on the social epistemology of disagreement and contrarianism not influenced (as far as I know) by the Overcoming Bias and Less Wrong conversations about the topic, see Christensen (2009); Ericsson et al. (2006); Kuchar (forthcoming); Miller (2013); Gelman (2009); Martin & Richards (1995); Schwed & Bearman (2010); Intemann & de Melo-Martin (2013). Also see Wikipedia’s article on scientific consensus.  ↩

3. I suppose I should mention that my entire inquiry here is, ala Goldman (1998), premised on the assumptions that (1) the point of epistemology is the pursuit of correspondence-theory truth, and (2) the point of social epistemology is to evaluate which social institutions and practices have instrumental value for producing true or well-calibrated beliefs.  ↩

4. I borrow this line from Chalmers (2014): “For much of the paper I am largely saying the obvious, but sometimes the obvious is worth saying so that less obvious things can be said from there.”  ↩

5. Holden Karnofsky seems to agree: “I think effective altruism falls somewhere on the spectrum between ‘contrarian view’ and ‘unusual taste.’ My commitment to effective altruism is probably better characterized as ‘wanting/choosing to be an effective altruist’ than as ‘believing that effective altruism is correct.’”  ↩

6. Without such heuristics, we can also rather quickly arrive at contradictions. For example, the majority of scholars who specialize in Allah’s existence believe that Allah is the One True God, and the majority of scholars who specialize in Yahweh’s existence believe that Yahweh is the One True God. Consistency isn’t everything, but contradictions like this should still be a warning sign.  ↩

7. According to the PhilPapers Surveys, 72.8% of philosophers are atheists, 14.6% are theists, and 12.6% categorized themselves as “other.” If we look only at metaphysicians, atheism remains dominant at 73.7%. If we look only at analytic philosophers, we again see atheism at 76.3%. As for physicists: Larson & Witham (1997) found that 77.9% of physicists and astronomers are disbelievers, and Pew Research Center (2009) found that 71% of physicists and astronomers did not believe in a god.  ↩

8. Muller & Bostrom (forthcoming). “Future Progress in Artificial Intelligence: A Poll Among Experts.”  ↩

9. But, this is unclear. First, I haven’t read the forthcoming paper, so I don’t yet have the full results of the survey, along with all its important caveats. Second, distributions of expert opinion can vary widely between polls. For example, Schlosshauer et al. (2013) reports the results of a poll given to participants in a 2011 quantum foundations conference (mostly physicists). When asked “When will we have a working and useful quantum computer?”, 9% said “within 10 years,” 42% said “10–25 years,” 30% said “25–50 years,” 0% said “50–100 years,” and 15% said “never.” But when the exact same questions were asked of participants at another quantum foundations conference just two years later, Norsen & Nelson (2013) report, the distribution of opinion was substantially different: 9% said “within 10 years,” 22% said “10–25 years,” 20% said “25–50 years,” 21% said “50–100 years,” and 12% said “never.”  ↩

10. I say “they” in this paragraph, but I consider myself to be a plausible candidate for an “AGI impact expert,” in that I’m unusually familiar with the arguments and evidence typically brought to bear on questions of long-term AI outcomes. I also don’t have a uniquely good track record on predicting long-term AI outcomes, nor am I among the discovered “super forecasters.” I haven’t participated in IARPA’s forecasting tournaments myself because it would just be too time consuming. I would, however, very much like to see these super forecasters grouped into teams and tasked with forecasting longer-term outcomes, so that we can begin to gather scientific data on which psychological and computational methods result in the best predictive outcomes when considering long-term questions. Given how long it takes to acquire these data, we should start as soon as possible.  ↩

11. Weiss & Shanteau (2012) would call them “privileged experts.”  ↩

12. Beckstead’s “elite common sense” prior and my “mildly extrapolated elite opinion” method are epistemic notions that involve some kind idealization or extrapolation of opinion. One earlier such proposal in social epistemology was Habermas’ “ideal speech situation,” a situation of unlimited discussion between free and equal humans. See Habermas’ “Wahrheitstheorien” in Schulz & Fahrenbach (1973) or, for an English description, Geuss (1981), pp. 65–66. See also the discussion in Tucker (2003), pp. 502–504.  ↩

13. Beckstead calls his method the “elite common sense” prior. I’ve named my method differently for two reasons. First, I want to distinguish MEEO from Beckstead’s prior, since I’m using the method for a slightly different purpose. Second, I think “elite common sense” is a confusing term even for Beckstead’s prior, since there’s some extrapolation of views going on. But also, it’s only a “mild” extrapolation — e.g. we aren’t asking what elites would think if they knew everything, or if they could rewrite their cognitive software for better reasoning accuracy.  ↩

14. My rough impression is that among the people who seem to have thought long and hard about AGI outcomes, and seem to me to exhibit fairly good epistemic practices on most issues, my view on AGI outcomes is still an outlier in its pessimism about the likelihood of desirable outcomes. But it’s hard to tell: there haven’t been systematic surveys of the important-to-me experts on the issue. I also wonder whether my views about long-term AGI outcomes are more a matter of seriously tackling a contrarian question rather than being a matter of having a particularly contrarian view. On this latter point, see this Facebook discussion.  ↩

15. I haven’t seen a poll of cryobiologists on the likely future technological feasibility of cryonics. Even if there were such polls, I’d wonder whether cryobiologists also had the relevant philosophical and neuroscientific expertise. I should mention that I’m not personally signed up for cryonics, for these reasons.  ↩

Discussion article for the meetup : Saint Petersburg, Russia

Discussion article for the meetup : Saint Petersburg, Russia

http://www.ietf.org/mail-archive/web/ietf/current/msg80472.html

Prominently mentioned in
http://lesswrong.com/lw/w8/engelbart_insufficiently_recursive/