Filter Last three months

Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Is Caviar a Risk Factor For Being a Millionaire?

39 Anders_H 09 December 2016 04:27PM

Today, my paper "Is caviar a risk factor for being a millionaire?" was published in the Christmas Edition of the BMJ (formerly the British Medical Journal).  The paper is available at but it is unfortunately behind a paywall. I am hoping to upload an open access version to a preprint server but this needs to be confirmed with the journal first. 

In this paper, I argue that the term "risk factor" is ambiguous, and that this ambiguity causes pervasive methodological confusion in the epidemiological literature. I argue that many epidemiological papers essentially use an audio recorder to determine whether a tree falling in the forest makes a sound, without being clear about which definition of "sound" they are considering.

Even worse, I argue that epidemiologists often try to avoid claiming that their results say anything about causality, by hiding behind "prediction models". When they do this. they often still control extensively for "confounding", a term which only has a meaning in causal models. I argue that this is analogous to stating that you are interested in whether trees falling in the forest causes any human to perceive the qualia of hearing, and then spending your methods section discussing whether the audio recorder was working properly.

Due to space constraints and other considerations, I am unable to state these analogies explicitly in the paper, but it does include a call for a taboo on the word risk factor, and a reference to Rationality: AI to Zombies. To my knowledge, this is the first reference to the book in the medical literature.

I will give a short talk about this paper at the Less Wrong meetup at the MIRI/CFAR office in Berkeley at 6:30pm tonight.

(I apologize for this short, rushed announcement, I was planning to post a full writeup but I was not expecting this paper to be published for another week) 

A Return to Discussion

33 sarahconstantin 27 November 2016 01:59PM

Epistemic Status: Casual

It’s taken me a long time to fully acknowledge this, but people who “come from the internet” are no longer a minority subculture.  Senators tweet and suburban moms post Minion memes. Which means that talking about trends in how people socialize on the internet is not a frivolous subject; it’s relevant to how people interact, period.

There seems to have been an overall drift towards social networks over blogs and forums in general, and in particular things like:

  • the drift of commentary from personal blogs to “media” aggregators like The AtlanticVox, and Breitbart
  • the migration of fandom from LiveJournal to Tumblr
  • Facebook and Twitter as the places where links and discussions go

At the moment I’m not empirically tracking any trends like this, and I’m not confident in what exactly the major trends are — maybe in future I’ll start looking into this more seriously. Right now, I have a sense of things from impression and hearsay.

But one thing I have noticed personally is that people have gotten intimidatedby more formal and public kinds of online conversation.  I know quite a few people who used to keep a “real blog” and have become afraid to touch it, preferring instead to chat on social media.  It’s a weird kind of perfectionism — nobody ever imagined that blogs were meant to be masterpieces.  But I do see people fleeing towards more ephemeral, more stream-of-consciousness types of communication, or communication that involves no words at all (reblogging, image-sharing, etc.)  There seems to be a fear of becoming too visible as a distinctive writing voice.

For one rather public and hilarious example, witness Scott Alexander’s  flight from LessWrong to LiveJournal to a personal blog to Twitter and Tumblr, in hopes that somewhere he can find a place isolated enough that nobody will notice his insight and humor. (It hasn’t been working.)

What might be going on here?

Of course, there are pragmatic concerns about reputation and preserving anonymity. People don’t want their writing to be found by judgmental bosses or family members.  But that’s always been true — and, at any rate, social networking sites are often less anonymous than forums and blogs.

It might be that people have become more afraid of trolls, or that trolling has gotten worse. Fear of being targeted by harassment or threats might make people less open and expressive.  I’ve certainly heard many writers say that they’ve shut down a lot of their internet presence out of exhaustion or literal fear.  And I’ve heard serious enough horror stories that I respect and sympathize with people who are on their guard.

But I don’t think that really explains why one would drift towards more ephemeral media. Why short-form instead of long-form?  Why streaming feeds instead of searchable archives?  Trolls are not known for their patience and rigor.  Single tweets can attract storms of trolls.  So troll-avoidance is not enough of an explanation, I think.

It’s almost as though the issue were accountability.  

A blog is almost a perfect medium for personal accountability. It belongs to you, not your employer, and not the hivemind.  The archives are easily searchable. The posts are permanently viewable. Everything embarrassing you’ve ever written is there.  If there’s a comment section, people are free to come along and poke holes in your posts. This leaves people vulnerable in a certain way. Not just to trolls, but to critics.

You can preempt embarrassment by declaring that you’re doing something shitty anyhow. That puts you in a position of safety. I think that a lot of online mannerisms, like using all-lowercase punctuation, or using really self-deprecating language, or deeply nested meta-levels of meme irony, are ways of saying “I’m cool because I’m not putting myself out there where I can be judged.  Only pompous idiots are so naive as to think their opinions are actually valuable.”

Here’s another angle on the same issue.  If you earnestly, explicitly say what you think, in essay form, and if your writing attracts attention at all, you’ll attract swarms of earnest, bright-but-not-brilliant, mostly young white male, commenters, who want to share their opinions, because (perhaps naively) they think their contributions will be welcomed. It’s basically just “oh, are we playing a game? I wanna play too!”  If you don’t want to play with them — maybe because you’re talking about a personal or highly technical topic and don’t value their input, maybe because your intention was just to talk to your friends and not the general public, whatever — you’ll find this style of interaction aversive.  You’ll read it as sealioning. Or mansplaining.  Or“well, actually”-ing.

I think what’s going on with these kinds of terms is something like:

Author: “Hi! I just said a thing!”

Commenter: “Ooh cool, we’re playing the Discussion game! Can I join?  Here’s my comment!”  (Or, sometimes, “Ooh cool, we’re playing the Verbal Battle game!  I wanna play! Here’s my retort!”)

Author: “Ew, no, I don’t want to play with you.”

There’s a bit of a race/gender/age/educational slant to the people playing the “commenter” role, probably because our society rewards some people more than others for playing the discussion game.  Privileged people are more likely to assume that they’re automatically welcome wherever they show up, which is why others tend to get annoyed at them.

Privileged people, in other words, are more likely to think they live in a high-trust society, where they can show up to strangers and be greeted as a potential new friend, where open discussion is an important priority, where they can trust and be trusted, since everybody is playing the “let’s discuss interesting things!” game.

The unfortunate reality is that most of the world doesn’t look like that high-trust society.

On the other hand, I think the ideal of open discussion, and to some extent the past reality of internet discussion, is a lot more like a high-trust society where everyone is playing the “discuss interesting things” game, than it is like the present reality of social media.

A lot of the value generated on the 90’s and early 2000’s internet was built on people who were interested in things, sharing information about those things with like-minded individuals.  Think of the websites that were just catalogues of information about someone’s obsessions. (I remember my family happily gathering round the PC when I was a kid, to listen to all the national anthems of the world, which some helpful net denizen had collated all in one place.)  There is an enormous shared commons that is produced when people are playing the “share info about interesting stuff” game.  Wikipedia. StackExchange. It couldn’t have been motivated by pure public-spiritedness — otherwise people wouldn’t have produced so much free work.  There are lower motivations: the desire to show off how clever you are, the desire to be a know-it-all, the desire to correct other people.  And there are higher motivations — obsession, fascination, the delight of infodumping. This isn’t some higher plane of civic virtue; it’s just ordinary nerd behavior.

But in ordinary nerd behavior, there are some unusual strengths.  Nerds are playing the “let’s have discussions!” game, which means that they’re unembarrassed about sharing their take on things, and unembarrassed about holding other people accountable for mistakes, and unembarrassed about being held accountable for mistakes.  It’s a sort of happy place between perfectionism and laxity.  Nobody is supposed to get everything right on the first try; but you’re supposed to respond intelligently to criticism. Things will get poked at, inevitably.  Poking is friendly behavior. (Which doesn’t mean it’s not also aggressive behavior.  Play and aggression are always intermixed.  But it doesn’t have to be understood as scary, hostile, enemy.)

Nerd-format discussions are definitely not costless. You get discussions of advanced/technical topics being mobbed by clueless opinionated newbies, or discussions of deeply personal issues being hassled by clueless opinionated randos.  You get endless debate over irrelevant minutiae. There are reasons why so many people flee this kind of environment.

But I would say that these disadvantages are necessary evils that, while they might be possible to mitigate somewhat, go along with having a genuinely public discourse and public accountability.

We talk a lot about social media killing privacy, but there’s also a way in which it kills publicness, by allowing people to curate their spaces by personal friend groups, and retreat from open discussions.   In a public square, any rando can ask an aristocrat to explain himself.  If people hide from public squares, they can’t be exposed to Socrates’ questions.

I suspect that, especially for people who are even minor VIPs (my level of online fame, while modest, is enough to create some of this effect), it’s tempting to become less available to the “public”, less willing to engage with strangers, even those who seem friendly and interesting.  I think it’s worth fighting this temptation.  You don’t get the gains of open discussion if you close yourself off.  You may not capture all the gains yourself, but that’s how the tragedy of the commons works; a bunch of people have to cooperate and trust if they’re going to build good stuff together.  And what that means, concretely, on the margin, is taking more time to explain yourself and engage intellectually with people who, from your perspective, look dumb, clueless, crankish, or uncool.

Some of the people I admire most, including theoretical computer scientist Scott Aaronson, are notable for taking the time to carefully debunk crackpots (and offer them the benefit of the doubt in case they are in fact correct.)  Is this activity a great ROI for a brilliant scientist, from a narrowly selfish perspective?  No. But it’s praiseworthy, because it contributes to a truly open discussion. If scientists take the time to investigate weird claims from randos, they’re doing the work of proving that science is a universal and systematic way of thinking, not just an elite club of insiders.  In the long run, it’s very important that somebody be doing that groundwork.

Talking about interesting things, with friendly strangers, in a spirit of welcoming open discussion and accountability rather than fleeing from it, seems really underappreciated today, and I think it’s time to make an explicit push towards building places online that have that quality.

In that spirit, I’d like to recommend LessWrong to my readers. For those not familiar with it, it’s a discussion forum devoted to things like cognitive science, AI, and related topics, and, back in its heyday a few years ago, it was suffused with the nerdy-discussion-nature. It had all the enthusiasm of late-night dorm-room philosophy discussions — except that some of the people you’d be having the discussions with were among the most creative people of our generation.  These days, posting and commenting is a lot sparser, and the energy is gone, but I and some other old-timers are trying to rekindle it. I’m crossposting all my blog posts there from now on, and I encourage everyone to check out and join the discussions there.

(Cross-posted from my blog,

How does personality vary across US cities?

30 JonahSinick 20 December 2016 08:00AM

In 2007, psychology researchers Michal Kosinski and David Stillwell released a personality testing app on Facebook app called myPersonality. The app ended up being used by 4 million Facebook users, most of whom consented to their personality question answers and some information from their Facebook profiles to be used for research purposes.

The very large sample size and matching data from Facebook profiles make it possible to investigate many questions about personality differences that were previously inaccessible. Koskinski and Stillwell have used it in a number of interesting publications, which I highly recommend (e.g. [1], [2] [3]).

In this post, I focus on what the dataset tells us about how big five personality traits vary by geographic region in the United States

continue reading »

Epistemic Effort

29 Raemon 29 November 2016 04:08PM

Epistemic Effort: Thought seriously for 5 minutes about it. Thought a bit about how to test it empirically. Spelled out my model a little bit. I'm >80% confident this is worth trying and seeing what happens. Spent 45 min writing post.

I've been pleased to see "Epistemic Status" hit a critical mass of adoption - I think it's a good habit for us to have. In addition to letting you know how seriously to take an individual post, it sends a signal about what sort of discussion you want to have, and helps remind other people to think about their own thinking.

I have a suggestion for an evolution of it - "Epistemic Effort" instead of status. Instead of "how confident you are", it's more of a measure of "what steps did you actually take to make sure this was accurate?" with some examples including:

  • Thought about it musingly
  • Made a 5 minute timer and thought seriously about possible flaws or refinements
  • Had a conversation with other people you epistemically respect and who helped refine it
  • Thought about how to do an empirical test
  • Thought about how to build a model that would let you make predictions about the thing
  • Did some kind of empirical test
  • Did a review of relevant literature
  • Ran an Randomized Control Trial
[Edit: the intention with these examples is for it to start with things that are fairly easy to do to get people in the habit of thinking about how to think better, but to have it quickly escalate to "empirical tests, hard to fake evidence and exposure to falsifiability"]

A few reasons I think this (most of these reasons are "things that seem likely to me" but which I haven't made any formal effort to test - they come from some background in game design and reading some books on habit formation, most of which weren't very well cited)
  • People are more likely to put effort into being rational if there's a relatively straightforward, understandable path to do so
  • People are more likely to put effort into being rational if they see other people doing it
  • People are more likely to put effort into being rational if they are rewarded (socially or otherwise) for doing so.
  • It's not obvious that people will get _especially_ socially rewarded for doing something like "Epistemic Effort" (or "Epistemic Status") but there are mild social rewards just for doing something you see other people doing, and a mild personal reward simply for doing something you believe to be virtuous (I wanted to say "dopamine" reward but then realized I honestly don't know if that's the mechanism, but "small internal brain happy feeling")
  • Less Wrong etc is a more valuable project if more people involved are putting more effort into thinking and communicating "rationally" (i.e. making an effort to make sure their beliefs align with the truth, and making sure to communicate so other people's beliefs align with the truth)
  • People range in their ability / time to put a lot of epistemic effort into things, but if there are easily achievable, well established "low end" efforts that are easy to remember and do, this reduces the barrier for newcomers to start building good habits. Having a nice range of recommended actions can provide a pseudo-gamified structure where there's always another slightly harder step you available to you.
  • In the process of writing this very post, I actually went from planning a quick, 2 paragraph post to the current version, when I realized I should really eat my own dogfood and make a minimal effort to increase my epistemic effort here. I didn't have that much time so I did a couple simpler techniques. But even that I think provided a lot of value.
Results of thinking about it for 5 minutes.

  • It occurred to me that explicitly demonstrating the results of putting epistemic effort into something might be motivational both for me and for anyone else thinking about doing this, hence this entire section. (This is sort of stream of conscious-y because I didn't want to force myself to do so much that I ended up going 'ugh I don't have time for this right now I'll do it later.')
  • One failure mode is that people end up putting minimal, token effort into things (i.e. randomly tried something on a couple doubleblinded people and call it a Randomized Control Trial).
  • Another is that people might end up defaulting to whatever the "common" sample efforts are, instead of thinking more creatively about how to refine their ideas. I think the benefit of providing a clear path to people who weren't thinking about this at all outweights people who might end up being less agenty about their epistemology, but it seems like something to be aware of.
  • I don't think it's worth the effort to run a "serious" empirical test of this, but I do think it'd be worth the effort, if a number of people started doing this on their posts, to run a followup informal survey asking "did you do this? Did it work out for you? Do you have feedback."
  • A neat nice-to-have, if people actually started adopting this and it proved useful, might be for it to automatically appear at the top of new posts, along with a link to a wiki entry that explained what the deal was.

Next actions, if you found this post persuasive:

Next time you're writing any kind of post intended to communicate an idea (whether on Less Wrong, Tumblr or Facebook), try adding "Epistemic Effort: " to the beginning of it. If it was intended to be a quick, lightweight post, just write it in its quick, lightweight form.

After the quick, lightweight post is complete, think about whether it'd be worth doing something as simple as "set a 5 minute timer and think about how to refine/refute the idea". If not, just write "thought about it musingly" after Epistemic Status. If so, start thinking about it more seriously and see where it leads.

While thinking about it for 5 minutes, some questions worth asking yourself:
  • If this were wrong, how would I know?
  • What actually led me to believe this was a good idea? Can I spell that out? In how much detail?
  • Where might I check to see if this idea has already been tried/discussed?
  • What pieces of the idea might you peel away or refine to make the idea stronger? Are there individual premises you might be wrong about? Do they invalidate the idea? Does removing them lead to a different idea? 

The Adventure: a new Utopia story

23 Stuart_Armstrong 25 December 2016 11:51AM

For an introduction to this story, see here. For a previous utopian attempt, see here. This story only explores a tiny part of this utopia.


The Adventure

Hark! the herald daemons spam,

Glory to the newborn World,

Joyful, all post-humans, rise,

Join the triumph of the skies.

Veiled in wire the Godhead see,

Built that man no more may die,

Built to raise the sons of earth,

Built to give them second birth.


The cold cut him off from his toes, then fingers, then feet, then hands. Clutched in a grip he could not unclench, his phone beeped once. He tried to lift a head too weak to rise, to point ruined eyes too weak to see. Then he gave up.

So he never saw the last message from his daughter, reporting how she’d been delayed at the airport but would be the soon, promise, and did he need anything, lots of love, Emily. Instead he saw the orange of the ceiling become blurry, that particularly hateful colour filling what was left of his sight.

His world reduced to that orange blur, the eternally throbbing sore on his butt, and the crisp tick of a faraway clock. Orange. Pain. Tick. Orange. Pain. Tick.

He tried to focus on his life, gather some thoughts for eternity. His dry throat rasped - another flash of pain to mingle with the rest - so he certainly couldn’t speak words aloud to the absent witnesses. But he hoped that, facing death, he could at least put together some mental last words, some summary of the wisdom and experience of years of living.

But his memories were denied him. He couldn’t remember who he was - a name, Grant, was that it? How old was he? He’d loved and been loved, of course - but what were the details? The only thought he could call up, the only memory that sometimes displaced the pain, was of him being persistently sick in a broken toilet. Was that yesterday or seventy years ago?

Though his skin hung loose on nearly muscle-free bones, he felt it as if it grew suddenly tight, and sweat and piss poured from him. Orange. Pain. Tick. Broken toilet. Skin. Orange. Pain...

The last few living parts of Grant started dying at different rates.


Much later:

continue reading »

Making intentions concrete - Trigger-Action Planning

23 Kaj_Sotala 01 December 2016 08:34PM

I'll do it at some point.

I'll answer this message later.

I could try this sometime.

For most people, all of these thoughts have the same result. The thing in question likely never gets done - or if it does, it's only after remaining undone for a long time and causing a considerable amount of stress. Leaving the "when" ambiguous means that there isn't anything that would propel you into action.

What kinds of thoughts would help avoid this problem? Here are some examples:

  • When I find myself using the words "later" or "at some point", I'll decide on a specific time when I'll actually do it.
  • If I'm given a task that would take under five minutes, and I'm not in a pressing rush, I'll do it right away.
  • When I notice that I'm getting stressed out about something that I've left undone, I'll either do it right away or decide when I'll do it.
Picking a specific time or situation to serve as the trigger of the action makes it much more likely that it actually gets done.

Could we apply this more generally? Let's consider these examples:
  • I'm going to get more exercise.
  • I'll spend less money on shoes.
  • I want to be nicer to people.
These goals all have the same problem: they're vague. How will you actually implement them? As long as you don't know, you're also going to miss potential opportunities to act on them.

Let's try again:
  • When I see stairs, I'll climb them instead of taking the elevator.
  • When I buy shoes, I'll write down how much money I've spent on shoes this year.
  • When someone does something that I like, I'll thank them for it.
These are much better. They contain both a concrete action to be taken, and a clear trigger for when to take it.

Turning vague goals into trigger-action plans

Trigger-action plans (TAPs; known as "implementation intentions" in the academic literature) are "when-then" ("if-then", for you programmers) rules used for behavior modification [i]. A meta-analysis covering 94 studies and 8461 subjects [ii] found them to improve people's ability for achieving their goals [iii]. The goals in question included ones such as reducing the amount of fat in one's diet, getting exercise, using vitamin supplements, carrying on with a boring task, determination to work on challenging problems, and calling out racist comments. Many studies also allowed the subjects to set their own, personal goals.

TAPs were found to work both in laboratory and real-life settings. The authors of the meta-analysis estimated the risk of publication bias to be small, as half of the studies included were unpublished ones.

Designing TAPs

TAPs work because they help us notice situations where we could carry out our intentions. They also help automate the intentions: when a person is in a situation that matches the trigger, they are much more likely to carry out the action. Finally, they force us to turn vague and ambiguous goals into more specific ones.

A good TAP fulfills three requirements [iv]:
  • The trigger is clear. The "when" part is a specific, visible thing that's easy to notice. "When I see stairs" is good, "before four o'clock" is bad (when before four exactly?). [v]
  • The trigger is consistent. The action is something that you'll always want to do when the trigger is fulfilled. "When I leave the kitchen, I'll do five push-ups" is bad, because you might not have the chance to do five push-ups each time when you leave the kitchen. [vi]
  • The TAP furthers your goals. Make sure the TAP is actually useful!
However, there is one group of people who may need to be cautious about using TAPs. One paper [vii] found that people who ranked highly on so-called socially prescribed perfectionism did worse on their goals when they used TAPs. These kinds of people are sensitive to other people's opinions about them, and are often highly critical of themselves. Because TAPs create an association between a situation and a desired way of behaving, it may make socially prescribed perfectionists anxious and self-critical. In two studies, TAPs made college students who were socially prescribed perfectionists (and only them) worse at achieving their goals.

For everyone else however, I recommend adopting this TAP:

When I set myself a goal, I'll turn it into a TAP.

Origin note

This article was originally published in Finnish at It draws heavily on CFAR's material, particularly the workbook from CFAR's November 2014 workshop.


[i] Gollwitzer, P. M. (1999). Implementation intentions: strong effects of simple plans. American psychologist, 54(7), 493.

[ii] Gollwitzer, P. M., & Sheeran, P. (2006). Implementation intentions and goal achievement: A meta‐analysis of effects and processes. Advances in experimental social psychology, 38, 69-119.

[iii] Effect size d = .65, 95% confidence interval [.6, .7].

[iv] Gollwitzer, P. M., Wieber, F., Myers, A. L., & McCrea, S. M. (2010). How to maximize implementation intention effects. Then a miracle occurs: Focusing on behavior in social psychological theory and research, 137-161.

[v] Wieber, Odenthal & Gollwitzer (2009; unpublished study, discussed in [iv]) tested the effect of general and specific TAPs on subjects driving a simulated car. All subjects were given the goal of finishing the course as quickly as possible, while also damaging their car as little as possible. Subjects in the "general" group were additionally given the TAP, "If I enter a dangerous situation, then I will immediately adapt my speed". Subjects in the "specific" group were given the TAP, "If I see a black and white curve road sign, then I will immediately adapt my speed". Subjects with the specific TAP managed to damage their cars less than the subjects with the general TAP, without being any slower for it.

[vi] Wieber, Gollwitzer, et al. (2009; unpublished study, discussed in [iv]) tested whether TAPs could be made even more effective by turning them into an "if-then-because" form: "when I see stairs, I'll use them instead of taking the elevator, because I want to become more fit". The results showed that the "because" reasons increased the subjects' motivation to achieve their goals, but nevertheless made TAPs less effective.

The researchers speculated that the "because" might have changed the mindset of the subjects. While an "if-then" rule causes people to automatically do something, "if-then-because" leads people to reflect upon their motivates and takes them from an implementative mindset to a deliberative one. Follow-up studies testing the effect of implementative vs. deliberative mindsets on TAPs seemed to support this interpretation. This suggests that TAPs are likely to work better if they can be carried out as consistently and as with little thought as possible.

[vii] Powers, T. A., Koestner, R., & Topciu, R. A. (2005). Implementation intentions, perfectionism, and goal progress: Perhaps the road to hell is paved with good intentions. Personality and Social Psychology Bulletin, 31(7), 902-912.

A review of cryonics/brain preservation in 2016

21 Andy_McKenzie 31 December 2016 06:19PM

Relevance to Less Wrong: Whether you think it is for better or worse, users on LW are about 50,000x more likely to be signed up for cryonics than the average person

Disclaimer: I volunteer at the Brain Preservation Foundation, but I speak for myself in this post and I'm only writing about publicly available information.

In 2016, cryonics remains a fringe operation. When it is discussed in the news or on social media, many express surprise that cryonics is a "real thing" outside of science fiction. Many others who do know about cryonics tend to label it a pseudoscience. Brain preservation (BP) through non-conventional cryonics methods such as those using aldehyde fixation is even more fringe, with most people not aware of it, and others dismissing it because it uses "toxic" chemicals. 

Here's a rundown of some events important to cryonics/BP in 2016. 

Research progress

- The Brain Preservation Foundation prize was won in February by Robert McIntyre and Greg Fahy. Their winning technique uses glutaraldehyde fixation followed by glycerol cryoprotection (in addition to a step to improve blood-brain barrier permeability and several other components) and allows for the preservation of neural structure as verified by electron microscopy across the cortex. McIntyre has since started a company called Nectome in part to improve and refine this procedure.
- Aschwin de Wolf of Advanced Neural Biosciences announced in November at the CryoSuisse conference that Advanced Neural Biosciences has developed a method that reduces dehydration in rat brain vitrification by using "brain optimized cryoprotectants." There is no peer-reviewed data or more detailed procedure available as of yet, and viability of the tissue may be a concern. 

Legal progress

- In Canada, Keegan Macintosh and Carrie Wong are challenging the anti-cryonics laws in British Columbia
- A right-to-die law passed in Colorado. Although not directly relevant to cryonics, it increases the number of locations where it might be possible to start brain preservation procedures in a more controlled manner by taking advantage of physician-assisted suicide in a terminally ill patient. This has been described as "cryothanasia" and is controversial both within the cryonics community and outside of it. 
- As far as I know, cryonics and brain preservation remain illegal in France, China, and many other areas. 

Current Cryonics Organizations 

- Alcor
- Cryonics Institute 
- KrioRus. They are planning on moving to Tver, which is a few hours west of Moscow (see Bloomberg profile). 
- Oregon Cryonics. This year, they put a hold on allowing users to sign up through their member portal, with the organization pivoting towards research until they can focus on "some critical cryonics research" to validate their methods. OC was profiled by Vice in March
- TransTime. This small cryonics company in San Leandro is still active, and was profiled in a video in Fusion earlier this year
- Osiris. This is a new, for-profit company in Florida that has so far been controversial within the cryonics community, and was recently profiled in the Miami New Times.  
- There are other organizations that only do standby and/or cryoprotectant perfusion. 

Essays about cryonics

- Tim Urban's post at Wait But Why about cryonics has wonderful diagrams explaining concepts such as why many people consider death to be a process, not an event. Like most everything Urban writes, it went viral and is still being posted on social media.  
- Corey Pein's article at The Baffler focuses primarily on critiques of Alcor and in particular Max More. 
- In April, an essay by Rachel Nuwer at BBC considered what would happen if cryonics worked. 
- Neuroscientist Clive Coen critiqued cryonics in an essay at New Humanist in November. 
- In January, PZ Myers critiqued aldehyde stabilized cryopreservation as "wishful thinking" because it is not yet possible to upload the memories/behaviors of even a simple organism based on information extracted post-fixation. 

Cryonics in the news

- In April, a profile of Elaine Walker, who is signed up with Alcor, on CNBC led to a moderately large amount of press for cryonics. 
- In August, a profile of Steve Aoki in Rolling Stone, who is also signed up with Alcor, mentions his plan to do cryonics. 
- In November, by far the biggest news story of the year about cryonics (dominating almost all of the Google trends variance) was about a 14-year-old girl who wanted cryonics and who had to go to court to prevent her father from stopping it. The court allowed her to be cryopreserved following her legal death. This case and related issues were covered extensively in the Guardian and other British news outlets, sparking debate about cryonics generally in the UK. 

Sample means, how do they work?

21 Benquo 20 November 2016 09:04PM

You know how people make public health decisions about food fortification, and medical decisions about taking supplements, based on things like the Recommended Daily Allowance? Well, there's an article in Nutrients titled A Statistical Error in the Estimation of the Recommended Dietary Allowance for Vitamin D. This paper says the following about the info used to establish the US recommended daily allowance for vitamin D:

The correct interpretation of the lower prediction limit is that 97.5% of study averages are predicted to have values exceeding this limit. This is essentially different from the IOM’s conclusion that 97.5% of individuals will have values exceeding the lower prediction limit.

The whole point of looking at averages is that individuals vary a lot due to a bunch of random stuff, but if you take an average of a lot of individuals, that cancels out most of the noise, so the average varies hardly at all. How much variation there is from individual to individual determines the population variance. How much variation you'd expect in your average due to statistical noise from sample to sample determines what we call the variation of the sample mean.

When you look at frequentist statistical confidence intervals, they are generally expressing how big the ordinary range of variation is for your average. For instance, 90% of the time, your average will not be farther off from the "true" average than it is from the boundaries of your confidence interval. This is relevant for answering questions like, "does this trend look a lot bigger than you'd expect from random chance?" The whole point of looking at large samples is that the errors have a chance to cancel out, leading to a very small random variation in the mean, relative to the variation in the population. This allows us to be confident that even fairly small differences in the mean are unlikely to be due to random noise.

The error here, was taking the statistical properties of the mean, and assuming that they applied to the population. In particular, the IOM looked at the dose-response curve for vitamin D, and came up with a distribution for the average response to vitamin D dosage. Based on their data, if you did another study like theirs on new data, it ought to predict that 600 IU of vitamin D is enough for the average person 97.5% of the time.

They concluded from this that 97.5% of people get enough vitamin D from 600 IU.

This is not an arcane detail. This is confusing the attributes of a population, with the attributes of an average. This is bad. This is real, real bad. In any sane world, this is mathematical statistics 101 stuff. I can imagine that someone who's heard about a margin of error a lot doesn't understand this stuff, but anyone who has to actually use the term should understand this.

Political polling is a simple example. Let's say that a poll shows 48% of Americans voting for the Republican and 52% for the Democrat, with a 5% margin of error. This means that 95% of polls like this one are expected to have an average within 5 percentage points of the true average. This does not mean that 95% of individual Americans have somewhere between a 43% and 53% chance of voting for the Republican. Most of them are almost definitively decided on one candidate, or the other. The average does not behave the same as the population. That's how fundamental this error is – it's like saying that all voters are undecided because the population is split.

Remember the famous joke about how the average family has two and a half kids? It's a joke because no one actually has two and a half kids. That's how fundamental this error is – it's like saying that there are people who have an extra half child hopping around. And this error caused actual harm:

The public health and clinical implications of the miscalculated RDA for vitamin D are serious. With the current recommendation of 600 IU, bone health objectives and disease and injury prevention targets will not be met. This became apparent in two studies conducted in Canada where, because of the Northern latitude, cutaneous vitamin D synthesis is limited and where diets contribute an estimated 232 IU of vitamin D per day. One study estimated that despite Vitamin D supplementation with 400 IU or more (including dietary intake that is a total intake of 632 IU or more) 10% of participants had values of less than 50 nmol/L. The second study reported serum 25(OH)D levels of less than 50 nmol/L for 15% of participants who reported supplementation with vitamin D. If the RDA had been adequate, these percentages should not have exceeded 2.5%. Herewith these studies show that the current public health target is not being met.

Actual people probably got hurt because of this. Some likely died.

This is also an example of scientific journals serving their intended purpose of pointing out errors, but it should never have gotten this far. This is a send a coal-burning engine under the control of a drunk engineer into the Taggart tunnel when the ventilation and signals are broken level of negligence. I think of the people using numbers as the reliable ones, but that's not actually enough – you have to think with them, you have to be trying to get the right answer, you have to understand what the numbers mean.

I can imagine making this mistake in school, when it's low stakes. I can imagine making this mistake on my blog. I can imagine making this mistake at work if I'm far behind on sleep and on a very tight deadline. But if I were setting public health policy? If I were setting the official RDA? I'd try to make sure I was right. And I'd ask the best quantitative thinkers I know to check my numbers.

The article was published in 2014, and as far as I can tell, as of the publication of this blog post, the RDA is unchanged.

(Cross-posted from my personal blog.)

A quick note on weirdness points and Solstices [And also random other Solstice discussion]

19 Raemon 21 December 2016 05:29PM

Common knowledge is important. So I wanted to note:

Every year on Solstice feedback forms, I get concerns about songs like "The X days of X-Risk" or "When I Die" (featuring lines including 'they may freeze my body when I die'), that they are too weird and ingroupy and offputting to people who aren't super-nerdy-transhumanists

But I also get comments from people who know little about X-risk or cryonics or whatever who say "these songs are hilarious and awesome." Sunday Assemblies who have no connection to Less Wrong sing When I Die and it's a crowd favorite every year.

And my impression is that people are only really weirded out by these songs on behalf of other people who are only weirded out by them on behalf of other people. There might be a couple people who are genuinely offput the ideas but if so it's not super clear to me. I take very seriously the notion of making Solstice inclusive while retaining it's "soul", talk to lots of people about what they find alienating or weird, and try to create something that can resonate with as many people as possible.

So I want it to at least be clear: if you are personally actually offput by those songs for your own sake, that makes sense and I want to know about it, but if you're just worried about other people, I'm pretty confident you don't need to be. The songs are designed so you don't need to take them seriously if you don't want to.


Random note 1: I think the only line that's raised concern from some non-LW-ish people for When I Die is "I'd prefer to never die at all", and that's because it's literally putting words in people's mouths which aren't true for everyone. I mentioned that to Glen. We'll see if he can think of anything else

Random note 2: Reactions to more serious songs like "Five Thousand Years" seem generally positive among non-transhumanists, although sometimes slightly confused. The new transhumanist-ish song this year, Endless Light, has gotten overall good reviews.

Circles of discussion

18 ciphergoth 16 December 2016 04:35AM

On Wednesday I had lunch with Raph Levien, and came away with a picture of how a website that fostered the highest quality discussion might work.


  • It’s possible that the right thing is a quick fix to Less Wrong as it is; this is about exploring what could be done if we started anew.
  • If we decided to start anew, what the software should do is only one part of what would need to be decided; that’s the part I address here.
  • As Anna Salamon set out, the goal is to create a commons of knowledge, such that a great many people have read the same stuff. A system that tailored what you saw to your own preferences would have its own strengths but would work entirely against this goal.
  • I therefore think the right goal is to build a website whose content reflects the preferences of one person, or a small set of people. In what follows I refer to those people as the “root set”.
  • A commons needs a clear line between the content that’s in and the content that’s out. Much of the best discussion is on closed mailing lists; it will be easier to get the participation of time-limited contributors if there’s a clear line of what discussion we want them to have read, and it’s short.
  • However this alone excludes a lot of people who might have good stuff to add; it would be good to find a way to get the best of both worlds between a closed list and an open forum.
  • I want to structure discussion as a set of concentric circles. 
  • Discussion in the innermost circle forms part of the commons of knowledge all can be assumed to be familiar with; surrounding it are circles of discussion where the bar is progressively lower. With a slider, readers choose which circle they want to read.
  • Content from rings further out may be pulled inwards by the votes of trusted people.
  • Content never moves outwards except in the case of spam/abuse.
  • Users can create top-level content in further-out rings and allow the votes of other users to move it closer to the centre. Users are encouraged to post whatever they want in the outermost rings, to treat it as one would an open thread or similar; the best content will be voted inwards.
  • Trust in users flows through endorsements starting from the root set.

More specifics on what that vision might look like:

  • The site gives all content (posts, top-level comments, and responses) a star rating from 0 to 5 where 0 means “spam/abuse/no-one should see”.
  • The rating that content can receive is capped by the rating of the parent; the site will never rate a response higher than its parent, or a top-level comment higher than the post it replies to.
  • Users control a “slider” a la Slashdot which controls the level of content that they see: set to 4, they see only 4 and 5-star content.
  • By default, content from untrusted users gets two stars; this leaves a star for “unusually bad” (eg rude) and one for “actual spam or other abuse”.
  • Content ratings above 2 never go down, except to 0; they only go up. Thus, the content in these circles can grow but not shrink, to create a stable commons.
  • Since a parent’s rating acts as a cap on the highest rating a child can get, when a parent’s rating goes up, this can cause a child’s rating to go up too.
  • Users rate content on this 0-5 scale, including their own content; the site aggregates these votes to generate content ratings.
  • Users also rate other users on the same scale, for how much they are trusted to rate content.
  • There is a small set of “root” users whose user ratings are wholly trusted. Trust flows from these users using some attack resistant trust metric.
  • Trust in a particular user can always go down as well as up.
  • Only votes from the most trusted users will suffice to bestow the highest ratings on content.
  • The site may show more trusted users with high sliders lower-rated content specifically to ask them to vote on it, for instance if a comment is receiving high ratings from users who are one level below them in the trust ranking. This content will be displayed in a distinctive way to make this purpose clear.
  • Votes from untrusted users never directly affect content ratings, only what is shown to more trusted users to ask for a rating. Downvoting sprees from untrusted users will thus be annoying but ineffective.
  • The site may also suggest to more trusted users that they uprate or downrate particular users.
  • The exact algorithms by which the site rates content, hands trust to users, or asks users for moderation would probably want plenty of tweaking. Machine learning could help here. However, for an MVP something pretty simple would likely get the site off the ground easily.

2016: a year in review in science

17 Elo 11 December 2016 05:01AM

As another year comes around, and our solstice plans come to a head I want to review this year's great progress in science to follow on from last year's great review.

The general criteria is: World changing science, not politics.  That means a lot of space discoveries, a lot of technology, some groundbreaking biology, and sometimes new chemical materials.  There really are too many to list briefly.

With that in mind, below is the list:

Things that spring to mind when you ask people:

  • T3d printing organs and skin tissue
  • Baby born with 3 parents. link
  • AlphaGo VS Lee Sedol
  • Cryopreservation of a rabbit brain - Link
  • Majorana fermions discovered (possibly quantum computing applications)
  • SpaceX landed Falcon 9 at sea - Link
  • Gravitational waves deteced by LIGO
  • Quantum logic gate with 99% accuracy at Oxford
  • TensorFlow has been out just over a year now.  An open source neural net project.



Note: the whole thing is worth reading - I cherry picked a few really cool ones.

  • Astronomers identify IDCS 1426 as the most distant massive galaxy cluster yet discovered, at 10 billion light years from Earth.[4]
  • Mathematicians, as part of the Great Internet Mersenne Prime Search, report the discovery of a new prime number: "274,207,281 − 1"
  • The world's first 13 TB solid state drive (SSD) is announced, doubling the previous record for a commercially available SSD. link
  • A successful head transplant on a monkey by scientists in China is reported.
  • The University of New South Wales announces that it will begin human trials of the Phoenix99, a fully implantable bionic eye. Link
  • Scientists in the United Kingdom are given the go-ahead by regulators to genetically modify human embryos by using CRISPR-Cas9 and related techniques. Link
  • Scientists announce Breakthrough Starshot, a Breakthrough Initiatives program, to develop a proof-of-concept fleet of small centimeter-sized light sail spacecraft, named StarChip, capable of making the journey to Alpha Centauri, the nearest extrasolar star system, at speeds of 20% and 15% of the speed of light, taking between 20 and 30 years to reach the star system, respectively, and about 4 years to notify Earth of a successful arrival. Link
  • A new paper in Astrobiology suggests there could be a way to simplify the Drake equation, based on observations of exoplanets discovered in the last two decades. link
  • A detailed report by the National Academies of Sciences, Engineering, and Medicine finds no risk to human health from genetic modifications of food. Link
  • Researchers from Queensland's Department of Environment and Heritage Protection, and the University of Queensland jointly report that the Bramble Cay melomys is likely extinct, adding: "Significantly, this probably represents the first recorded mammalian extinction due to anthropogenic climate change." Link
  • Scientists announce detecting a second gravitational wave event (GW151226) resulting from the collision of black holes.   Link
  • The first known death caused by a self-driving car is disclosed by Tesla Motors. Link
  • A team at the University of Oxford achieves a quantum logic gate with record-breaking 99.9% precision, reaching the benchmark required to build a quantum computer. Link
  • The world's first baby born through a controversial new "three parent" technique is reported. Link
  • A team at Australia's University of New South Wales create a new quantum bit that remains in a stable superposition for 10 times longer than previously achieved. Link
  • Scientists at the International Union of Pure and Applied Chemistry officially recognizes names for four new chemical elements: Nihonium, Nh, 113; Moscovium, Mc, 115; Tennessine, Ts, 117 and Oganesson, Og, 118. Link


Notable deaths:

Nobel prizes:

  • The Nobel Prize in Chemistry 2016 was awarded jointly to Jean-Pierre Sauvage, Sir J. Fraser Stoddart and Bernard L. Feringa "for the design and synthesis of molecular machines"
  • The Nobel Prize in Physics 2016 was divided, one half awarded to David J. Thouless, the other half jointly to F. Duncan M. Haldane and J. Michael Kosterlitz "for theoretical discoveries of topological phase transitions and topological phases of matter".
  • The Nobel Prize in Physiology or Medicine 2016 was awarded to Yoshinori Ohsumi "for his discoveries of mechanisms for autophagy".
  • The Nobel Prize in Literature 2016 was awarded to Bob Dylan "for having created new poetic expressions within the great American song tradition".
  • The Nobel Peace Prize 2016 was awarded to Juan Manuel Santos "for his resolute efforts to bring the country's more than 50-year-long civil war to an end".
  • The Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel 2016 was awarded jointly to Oliver Hart and Bengt Holmström "for their contributions to contract theory"

100 years ago (1916):

Nobel Prizes in 1916:

  • Physics – not awarded
  • Chemistry – not awarded
  • Medicine – not awarded
  • Literature – Carl Gustaf Verner von Heidenstam
  • Peace – not awarded


  • Pokemon go
  • Brexit - Britain secedes from the EU
  • Donald trump US president
  • SpaceX making more launches, and had a major explosion setback
  • project delayed by SpaceX expolosion.

Meta: this took in the order of 3+ hours to write over several weeks.

Cross posted to Lesswrong here.

Downvotes temporarily disabled

17 Vaniver 01 December 2016 05:31PM

This is a stopgap measure until admins get visibility into comment voting, which will allow us to find sockpuppet accounts more easily.


The best place to track changes to the codebase is the github LW issues page.

[Link] On Trying Not To Be Wrong

17 sarahconstantin 11 November 2016 07:25PM

[Link] Dominic Cummings: how the Brexit referendum was won

16 The_Jaded_One 12 January 2017 09:26PM

The Partial Control Fallacy

16 Darmani 14 December 2016 02:57AM


Around the time I started grad school, I applied for a few prestigious fellowships. Winning is determined by several factors. Some are just an application, while some have a follow-up interview, but the applications all get scored on a rubric that looks roughly like this:
  • 50%: Past research
  • 30%: Letters of recommendation
  • 10%: Transcript
  • 10%: Personal Essays
Naturally, I proceeded to pour massive amounts of time into the essays, letting it consume much of my free time for the month of October.

Getting that Fellowship will really help me have a successful graduate career. Writing better essays will help me get the Fellowship. Therefore, to the extent that I care about having a successful graduate career, I should be willing to work hard on those essays.

But if the real goal is a successful graduate career, then at some point shouldn’t I put those essays down and do something else, like reading a few papers or practicing public speaking?

This, I dub the Partial Control Fallacy. It’s where, if there’s some outcome you want, and you only control a couple factors that affect that outcome, you decide how much to try to improve those factors as if you were actually improving the entire outcome. It’s closely connected to the 80/20 principle: it’s when you only have control over that last 20%, but you pretend it’s the whole thing and work on it accordingly. It’s when the 80/20 principle would suggest doing nothing at all.

Here are some more examples:
  • Trying to get any competitive award that’s judged mostly by your past. The best college application is stellar grades and some good awards, the best resume is a great network and lots of success stories, and the best pitch to VCs is a rock-solid business.
  • Thinking really hard about what to say to that cute guy or girl across the room. Most of what happens is determined before you open your mouth by what they’re looking for and whether they’re attracted to you.
  • Worrying about small optimizations when writing code, like avoiding copying small objects. Most of good performance comes from the high-level design of the system.
I think I’ve been guilty of all three of these at one point or another. I don’t want to think about how much time I spent on my Thiel Fellowship application and preparing for my YCombinator interview. Meanwhile, most people who get into either don’t spend much time at all.

In parallel computing, there’s a concept called Amdahl’s law. If your program takes tsteps to run, and you can make s steps faster by a factor of f (say, by splitting them across multiple processors), then the new speed is t-s+s/f, for a speedup of t/(t-s+s/f). Therefore, if you optimize those s steps extremely hard and split them across infinite cores, the best speedup you’ll get is t/(t-s).

Applying that to the above, and you can see that, if I worked infinitely hard on my essays, I could only make my apps 11% better versus not doing anything at all. (At least to the extent that it really does follow that rubric, even if I submit a blank essay.)

Sometimes, that outcome is all you care about it, in which case you’re perfectly justified in trying to eke out every advantage you can get. If you’re in a massively competitive field, like sports or finance, where there’s a really big difference between being #1 and being #2 at some narrow thing, then, by all means, go get that last 1%. Wake up early, get that 7th computer monitor, rinse your cottage cheese. But if you’re putting this kind of effort into something because it’s your terminal goal — well, you’re not doing this for anything else, are you?

I think the solution to this fallacy is always to think past the immediate goal. Instead of asking “How can I get this Fellowship,” ask “How can I improve my research career.” When you see the road ahead of you as just a path to your larger mission, something that once seemed like your only hope now becomes one option among many.

Planning the Enemy's Retreat

15 Gram_Stone 11 January 2017 05:44AM

Related: Leave a Line of Retreat

When I was smaller, I was sitting at home watching The Mummy, with my mother, ironically enough. There's a character by the name of Bernard Burns, and you only need to know two things about him. The first thing you need to know is that the titular antagonist steals his eyes and tongue because, hey, eyes and tongues spoil after a while you know, and it's been three thousand years.

The second thing is that Bernard Burns was the spitting image of my father. I was terrified! I imagined my father, lost and alone, certain that he would die, unable to see, unable even to properly scream!

After this frightening ordeal, I had the conversation in which it is revealed that fiction is not reality, that actions in movies don't really have consequences, that apparent consequences are merely imagined and portrayed.

Of course I knew this on some level. I think the difference between the way children and adults experience fiction is a matter of degree and not kind. And when you're an adult, suppressing those automatic responses to fiction has itself become so automatic, that you experience fiction as a thing compartmentalized. You always know that the description of consequences in the fiction will not by magic have fire breathed into them, that Imhotep cannot gently step out of the frame and really remove your real father's real eyes.

So, even though we often use fiction to engage, to make things feel more real, in another way, once we grow, I think fiction gives us the chance to entertain formidable ideas at a comfortable distance.

A great user once said, "Vague anxieties are powerful anxieties." Related to this is the simple rationality technique of Leaving a Line of Retreat: before evaluating the plausibility of a highly cherished or deeply frightening belief, one visualizes the consequences of the highly cherished belief being false, or of the deeply frightening belief being true. We hope that it will thereby become just a little easier to evaluate the plausibility of that belief, for if we are wrong, at least we know what we're doing about it. Sometimes, if not often, what you'd really do about it isn't as bad as your intuitions would have you think.

If I had to put my finger on the source of that technique's power, I would name its ability to reduce the perceived hedonic costs of truthseeking. It's hard to estimate the plausibility of a charged idea because you expect your undesired outcome to feel very bad, and we naturally avoid this. The trick is in realizing that, in any given situation, you have almost certainly overestimated how bad it would really feel.

But Sun Tzu didn't just plan his own retreats; he also planned his enemies' retreats. What if your interlocutor has not practiced the rationality technique of Leaving a Line of Retreat? Well, Sun Tzu might say, "Leave one for them."

As I noted in the beginning, adults automatically compartmentalize fiction away from reality. It is simply easier for me to watch The Mummy than it was when I was eight. The formidable idea of my father having his eyes and tongue removed is easier to hold at a distance.

Thus, I hypothesize, truth in fiction is hedonically cheap to seek.

When you recite the Litany of Gendlin, you do so because it makes seemingly bad things seem less bad. I propose that the idea generalizes: when you're experiencing fiction, everything seems less bad than its conceivably real counterpart, it's stuck inside the book, and any ideas within will then seem less formidable. The idea is that you can use fiction as an implicit line of retreat, that you can use it to make anything seem less bad by making it make-believe, and thus, safe. The key, though, is that not everything inside of fiction is stuck inside of fiction forever. Sometimes conclusions that are valid in fiction also turn out to be valid in reality. 

This is hard to use on yourself, because you can't make a real scary idea into fiction, or shoehorn your scary idea into existing fiction, and then make it feel far away. You'll know where the fiction came from. But I think it works well on others.

I don't think I can really get the point across in the way that I'd like without an example. This proposed technique was an accidental discovery, like popsicles or the Slinky:

A history student friend of mine was playing Fallout: New Vegas, and he wanted to talk to me about which ending he should choose. The conversation seemed mostly optimized for entertaining one another, and, hoping not to disappoint, I tried to intertwine my fictional ramblings with bona fide insights. The student was considering giving power to a democratic government, but he didn't feel very good about it, mostly because this fictional democracy was meant to represent anything that anyone has ever said is wrong with at least one democracy, plausible or not.

"The question you have to ask yourself," I proposed to the student, "is 'Do I value democracy because it is a good system, or do I value democracy per se?' A lot of people will admit that they value democracy per se. But that seems wrong to me. That means that if someone showed you a better system that you could verify was better, you would say 'This is good governance, but the purpose of government is not good governance, the purpose of government is democracy.' I do, however, understand democracy as a 'current best bet' or local maximum."

I have in fact gotten wide-eyed stares for saying things like that, even granting the closing ethical injunction on democracy as local maximum. I find that unusual, because it seems like one of the first steps you would take towards thinking about politics clearly, to not equivocate democracy with good governance. If you were further in the past and the fashionable political system were not democracy but monarchy, and you, like many others, consider democracy preferable to monarchy, then upon a future human revealing to you the notion of a modern democracy, you would find yourself saying, regrettably, "This is good governance, but the purpose of government is not good governance, the purpose of government is monarchy."

But because we were arguing for fictional governments, our autocracies, or monarchies, or whatever non-democratic governments heretofore unseen, could not by magic have fire breathed into them. For me to entertain the idea of a non-democratic government in reality would have solicited incredulous stares. For me to entertain the idea in fiction is good conversation.

The student is one of two people with whom I've had this precise conversation, and I do mean in the particular sense of "Which Fallout ending do I pick?" I snuck this opinion into both, and both came back weeks later to tell me that they spent a lot of time thinking about that particular part of the conversation, and that the opinion I shared seemed deep.

Also, one of them told me that they had recently received some incredulous stares.

So I think this works, at least sometimes. It looks like you can sneak scary ideas into fiction, and make them seem just non-scary enough for someone to arrive at an accurate belief about that scary idea.

I do wonder though, if you could generalize this even more. How else could you reduce the perceived hedonic costs of truthseeking?

[Link] If we can't lie to others, we will lie to ourselves

15 paulfchristiano 26 November 2016 10:29PM

[Link] The engineer and the diplomat

14 Benquo 27 December 2016 08:49PM

Land war in Asia

14 Apprentice 07 December 2016 07:31PM

Introduction: Here's a misconception about World War II that I think is harmful and I don't see refuted often enough.

Misconception: In 1941, Hitler was sitting pretty with most of Europe conquered and no huge difficulties on the horizon. Then, due to his megalomania and bullshit ideology, he decided to invade Russia. This was an unforced error of epic proportions. It proved his undoing, like that of Napoleon before him.

Rebuttal: In hindsight, we think of the Soviet Union as a superpower and military juggernaut which you'd be stupid to go up against. But this is not how things looked to the Germans in 1941. Consider World War I. In 19171918, Germany and Austria had defeated Russia at the same time as they were fighting a horrifyingly bloody war with France and Britain - and another devastating European war with Italy. In 1941, Italy was an ally, France had been subdued and Britain wasn't in much of a position to exert its strength. Seemingly, the Germans had much more favorable conditions than in the previous round. And they won the previous round.

In addition, the Germans were not crazy to think that the Red Army was a bit of a joke. The Russians had had their asses handed to them by Poland in 1920 and in 19391940 it had taken the Russians three months and a ridiculous number of casualties to conquer a small slice of Finland.

Nevertheless, Russia did have a lot of manpower and a lot of equipment (indeed, far more than the Germans had thought) and was a potential threat. The Molotov-Ribbentrop pact was obviously cynical and the Germans were not crazy to think that they would eventually have to fight the Russians. Being the first to attack seemed like a good idea and 1941 seemed like a good time to do it. The potential gains were very considerable. Launching the invasion was a rational military decision.

Why this matters: The idea that Hitler made his most fatal decision for irrational reasons feeds into the conception that evil and irrationality must go hand in hand. It's the same kind of thinking that makes people think a superintelligence would automatically be benign. But there is no fundamental law of the universe which prevents a bad guy from conquering the world. Hitler lost his war with Russia for perfectly mundane and contingent reasons like, “the communists had been surprisingly effective at industrialization.”

Triaging mental phenomena or: leveling up the stack trace skill

13 RomeoStevens 23 December 2016 12:15AM

Related: The 5-Second LevelAttention control is critical for increasing/altering motivation

Epistemic Status: sharing a hypothesis that's slowly been coalescing since a discussion with Eliezer at EAG and got catalyzed by Anna's latest LW post along with an exercise I have been using. n=1

Mental phenomena (and thus rationality skills) can't be trained without a feedback loop that causes calibration in the relevant direction. One of my guesses for a valuable thing Eliezer did was habitual stack traces causing a leveling up of stack trace resolution i.e. seeing more fine grain detail in mental phenomena. This is related to 'catching flinches' as Anna describes, as an example of a particularly useful phenomena to be able to catch. In general, you can't tune black boxes, you need to be able to see individual steps.

How can you level up the stack trace skill? Triaging your unwillingness to do things, and we'll start with your unwillingness to practice the stack trace skill! I like 'triage' more than 'classify' because it imports some connotations about scope sensitivity.

In order to triage we need a taxonomy. Developing/hacking/modding your own is what ultimately works best, but you can use prebuilt ones as training wheels. Here are two possible taxonomies:

Note whether it is experienced as

  • Distracting Desire
  • Aversion
  • Laziness
  • Agitation/Annoyance
  • Ambiguity/Doubt

Note whether it is experienced as 

  • Mental talk
  • Mental images
  • Sensations in the body

Form the intention to practice the stack trace skill and then try to classify at least one thing that happens. If you feel good when you get a 'hit' you will be more likely to catch additional events.

You can try this on anything. The desire for an unhealthy snack, the unwillingness to email someone etc. Note that the exercise isn't about forcing yourself to do things you don't want to do. You just want to see more clearly your own objections to doing it. If you do it more, you'll start to notice that you can catch more 'frames' or multiple phenomena at the same time or in a row e.g. I am experiencing ambiguity as the mental talk "I'm not sure how to do that" and as a slightly slimy/sliding away sensation followed by aversion to feeling the slimy feeling and an arising distracting desire to check my email. Distinguishing between actual sensations in the body and things that only seem like they could maybe be described as sensations is mostly a distraction and not all that important initially.

These are just examples and finding nice tags in your own mentalese makes the thing run smoother. You can also use this as fuel for focusing for particularly interesting frames you catch e.g. when you catch a limiting belief. It's also interesting to notice instances of the 'to-be' verb form in mental talk as this is the source of a variety of map-territory distinction errors.

There is a specific failure worth mentioning: coming up with a story. If you ask yourself questions like "Why did I think that?" your brain is great at coming up with plausible sounding stories that are often bullshit. This is why, when practicing the skill, you have to prime the intention to catch specific things beforehand. Once the skill has been built up you can use it on arbitrary thoughts and have a sense for the difference between 'story' and actual frame catching.

If other people try this I'm curious for feedback. My experience so far has been that increasing the resolution on stack traces has made the practice of every other mental technique dramatically easier because the feedback loops are all tighter. Especially relevant to repairing a failed TAP. How much practice was involved? A few minutes a day for 3 weeks caused a noticeable effect that has endured. My models, plans, and execution fail less often. When they do I have a much better chance of catching the real culprit.

Improve comments by tagging claims

13 Benquo 20 December 2016 05:04PM

I used to think that comments didn’t matter. I was wrong. This is important because communities of discourse are an important source of knowledge. I’ll explain why I changed my mind, and then propose a simple mechanism for improving them, that can be implemented on any platform that allows threaded comments.

continue reading »

Which areas of rationality are underexplored? - Discussion Thread

13 casebash 01 December 2016 10:05PM

There seems to actually be real momentum behind this attempt as reviving Less Wrong. One of the oldest issues on LW has been the lack of content. For this reason, I thought that it might be worthwhile opening a thread where people can suggest how we can expand the scope of what people write about in order for us to have sufficient content.

Does anyone have any ideas about which areas of rationality are underexplored? Please only list one area per comment.

What's the most annoying part of your life/job?

13 Liron 23 October 2016 03:37AM

Hi, I'm an entrepreneur looking for a startup idea.

In my experience, the reason most startups fail is because they never actually solve anyone's problem. So I'm cheating and starting out by identifying a specific person with a specific problem.

So I'm asking you, what's the most annoying part of your life/job? Also, how much would you pay for a solution?

Project Hufflepuff

12 Raemon 18 January 2017 06:57PM

(This is a crossposted FB post, so it might read a bit weird)

My goal this year (in particular, my main focus once I arrive in the Bay, but also my focus in NY and online in the meanwhile), is to join and champion the growing cause of people trying to fix some systemic problems in EA and Rationalsphere relating to "lack of Hufflepuff virtue".

I want Hufflepuff Virtue to feel exciting and important, because it is, and I want it to be something that flows naturally into our pursuit of both epistemic integrity, intellectual creativity, and concrete action.

Some concrete examples:

- on the 5 second reflex level, notice when people need help or when things need doing, and do those things.

- have an integrated understanding that being kind to people is *part* of helping them (and you!) to learn more, and have better ideas.

(There are a bunch of ways to be kind to people that do NOT do this, i.e. politely agreeing to disagree. That's not what I'm talking about. We need to hold each other to higher standards but not talk down to people in a fashion that gets in the way of understanding. There are tradeoffs and I'm not sure of the best approach but there's a lot of room for improvement)

- be excited and willing to be the person doing the grunt work to make something happen

- foster a sense that the community encourages people to try new events, actively take personal responsibility to notice and fix community-wide problems that aren't necessarily sexy.

- when starting new projects, try to have mentorship and teamwork built into their ethos from the get-go, rather than hastily tacked on later

I want these sorts of things to come easily to mind when the future people of 2019 think about the rationality community, and have them feel like central examples of the community rather than things that we talk about wanting-more-of.

[Link] EA Has A Lying Problem

12 Benquo 11 January 2017 10:31PM

[Link] Yudkowsky's 'Four Layers of Intellectual Conversation'

12 Gram_Stone 08 January 2017 09:47PM

A different argument against Universal Basic Income

12 chaosmage 28 December 2016 10:35PM

I grew up in socialist East Germany. Like most of my fellow citizens, I was not permitted to leave the country. But there was an important exception: People could leave after retirement. Why? Because that meant their forfeited their retirement benefits. Once you took more from the state than you gave, you were finally allowed to leave. West Germany would generously take you in. My family lived near the main exit checkpoint for a while and there was a long line of old people most days.

And then there is Saudi Arabia and other rentier states. Rentier states( derive most of their income not from their population. The population gets a lot more wealth from the state than the state gets from the population. States like Saudi Arabia are therefore relatively independent of their population's consent with policy. A citizen who is unhappy is welcome to leave, or to retreat to their private sphere and live off benefits while keeping their mouth shut - neither of these options incurs a significant cost for the state.

I think these facts are instructive in thinking about Universal Basic Income. I want to make a point that I haven't seen made in discussions of the matter.

Most political systems (not just democracies) are built on an assumption that the state needs its citizens. This assumption is always a bit wrong - for example, no state has much need of the terminally ill, except to signal to its citizens that it cares for all of them. In the cases of East Germany and Saudi Arabia, this assumption is more wrong. And Universal Basic Income makes it more wrong as well.

From the point of view of a state, there are citizens who are more valuable (or who help in competition with other states) and ones who are more of a burden (who make competing with other states more difficult). Universal Basic Income massively broadens the part of society that is a net loss to the state.

Now obviously technological unemployment is likely to do that anyway. But there's a difference between answers to that problem that divide up the available work between the members of society and answers that divide up society into contributors and noncontributors. My intuition is that UBI is the second kind of solution, because states will be incentivized to treat contributors differently from noncontributors. The examples are to illustrate that a state can behave very differently towards citizens if it is fundamentally not interested in retaining them.

I go along with Harari's suggestion that the biggest purely political problem of the 21st century is the integration of the economically unnecessary parts of the population into society. My worry is that UBI, while helping with immediate economic needs, makes that problem worse in the long run. Others have already pointed out problems with UBI (such as that in a democracy it'll be impossible to get rid of if it is a failure) that gradual approaches like lower retirement age, later entry into the workforce and less work per week don't have. But I reckon that behind the immediate problems with UBI such as the amount of funding it needs and the question of what it does to the motivation to work, there's a whole class of problems that arise out of the changed relationships between citizens, states and economies. With complex networks of individuals and institutions responding intelligently to the changed circumstances, a state inviting its citizens to emigrate may not be the weirdest of unforeseen consequences.

Ideas for Next Generation Prediction Technologies

12 ozziegooen 20 December 2016 10:06PM

Prediction markets are powerful, but also still quite niche. I believe that part of this lack of popularity could be solved with significantly better tools. During my work with Guesstimate I’ve thought a lot about this issue and have some ideas for what I would like to see in future attempts at prediction technologies.



1. Machine learning for forecast aggregation

In financial prediction markets, the aggregation method is the market price. In non-market prediction systems, simple algorithms are often used. For instance, in the Good Judgement Project, the consensus trends displays “the median of the most recent 40% of the current forecasts from each forecaster.”[1] Non-financial prediction aggregation is a pretty contested field with several proposed methods.[2][3][4]

I haven’t heard much about machine learning used for forecast aggregation. It would seem to me like many, many factors could be useful in aggregating forecasts. For instance, some elements of one’s social media profile may be indicative of their forecasting ability. Perhaps information about the educational differences between multiple individuals could provide insight on how correlated their knowledge is.

Perhaps aggregation methods, especially with training data, could partially detect and offset predictable human biases. If it is well known that people making estimates of project timelines are overconfident, then this could be taken into account. For instance, someone enters in “I think I will finish this project in 8 weeks”, and the system can infer something like, “Well, given the reference class I have of similar people making similar calls, I’d expect it to take 12.

A strong machine learning system would of course require a lot of sample data, but small strides may be possible with even limited data. I imagine that if data is needed, lots of people on platforms like Mechanical Turk could be sampled.

2. Prediction interval input

The prediction tools I am familiar with focus on estimating the probabilities of binary events. This can be extremely limiting. For instance, instead of allowing users to estimate what Trump’s favorable rating would be, they instead have to bet on whether it will be over a specific amount, like “Will Trump’s favorable rate be at least 45.0% on December 31st?”[5]

It’s probably no secret that I have a love for probability densities. I propose that users should be able to enter probability densities directly. User entered probability densities would require more advanced aggregation techniques, but is doable.[6]

Probability density inputs would also require additional understanding from users. While this could definitely be a challenge, many prediction markets already are quite complicated, and existing users of these tools are quite sophisticated.

I would suspect that using probability densities could simplify questions about continuous variables and also give much more useful information on their predictions. If there are tail risks these would be obvious; and perhaps more interestingly, probability intervals from prediction tools could be directly used in further calculations. For instance, if there were separate predictions about the population of the US and the average income, these could be multiplied to have an estimate of the total GDP (correlations complicate this, but for some problems may not be much of an issue, and in others perhaps they could be estimated as well).

Probability densities make less sense for questions with a discrete set of options, like predicting who will win an election. There are a few ways of dealing with these. One is to simply leave these questions to other platforms, or to resort back to the common technique of users estimating specific percentage likelihoods in these cases. Another is to modify some of these to be continuous variables that determine discrete outcomes; like the number of electoral college votes a U.S. presidential candidate will receive. Another option is to estimate the ‘true’ probability of something as a distribution, where the ‘true’ probability is defined very specifically. For instance, a group could make probability density forecasts for the probability that the blog 538 will give to a specific outcome on a specific date. In the beginning of an election, people would guess 538's percent probability for one candidate winning a month before the election.

3. Intelligent Prize Systems

I think the main reason why so many academics and rationalists are excited about prediction markets is because of their positive externalities. Prediction markets like InTrade seem to do quite well at predicting many political and future outcomes, and this information is very valuable to outside third parties.

I’m not sure how comfortable I feel about the incentives here. The fact that the main benefits come from externalities indicates that the main players in the markets aren’t exactly optimizing for these benefits. While users are incentivized to be correct and calibrated, they are not typically incentivized to predict things that happen to be useful for observing third parties.

I would imagine that the externalities created by prediction tools would be strongly correlate with the value of information to these third parties, which does rely on actionable and uncertain decisions. So if the value of information from prediction markets were to be optimized, it would make sense that these third parties have some way of ranking what gets attention based on what their decisions are.


For instance, a whole lot of prediction markets and related tools focus heavily on sports forecasts. I highly doubt that this is why most prediction market enthusiasts get excited about these markets.

In many ways, promoting prediction markets for their positive externalities is very strange endeavor. It’s encouraging the creation of a marketplace because of the expected creation of some extra benefit that no one directly involved in that marketplace really cares about. Perhaps instead there should be otherwise-similar ways for those who desire information from prediction groups to directly pay for that information.

One possibility that has been discussed is for prediction markets to be subsidized in specific ways. This obviously would have to be done carefully in order to not distort incentives. I don’t recall seeing this implemented successfully yet, just hearing it be proposed.

For prediction tools that aren’t markets, prizes can be given out by sponsoring parties. A naive system is for one large sponsor to sponsor a ‘category’, then the best few people in that category get the prizes. I believe something like this is done by Hypermind.

I imagine a much more sophisticated system could pay people as they make predictions. One could imagine a system that numerically estimates how much information was added to the new aggregate when a new prediction is made. Users with established backgrounds will influence the aggregate forecast significantly more than newer ones, and thus will be rewarded proportionally. A more advanced system would also take into account estimate supply and demand; if there are some conditions where users particularly enjoy adding forecasts, they may not need to be compensated as much for these, despite the amount or value of information contributed.

On the prize side, a sophisticated system could allow various participants to pool money for different important questions and time periods. For instance, several parties put down a total of $10k on the question ‘what will the US GDP be in 2020’, to be rewarded over the period of 2016 to 2017. Participants who put money down could be rewarded by accessing that information earlier than others or having improved API access.

Using the system mentioned above, an actor could hypothetically build up a good reputation, and then use it to make a biased prediction in the expectation that it would influence third parties. While this would be very possible, I would expect it to require the user to generate more value than their eventual biased prediction would cost. So while some metrics may become somewhat biased, in order for this to happen many others would become improved. If this were still a problem, perhaps forecasts could make bets in order to demonstrate confidence (even if the bet were made in a separate application).

4. Non-falsifiable questions

Prediction tools are really a subset of estimation tools, where the requirement is that they estimate things that are eventually falsifiable. This is obviously a very important restriction, especially when bets are made. However, it’s not an essential restriction, and hypothetically prediction technologies could be used for much more general estimates.

To begin, we could imagine how very long term ideas could be forecasted. A simple model would be to have one set of forecasts for what the GDP will be in 2020, and another for what the systems’ aggregate will think the GDP is in 2020, at the time of 2018. Then in 2018 everyone could be ranked, even though the actual event has not yet occurred.

In order for the result in 2018 to be predictive it would obviously require that participants would expect future forecasts to be predictive. If participants thought everyone else would be extremely optimistic, they would be encouraged to make optimistic predictions as well. This leads to a feedback loop that the more accurate the system is thought to be the more accurate it will be (approaching the accuracy of an immediately falsifiable prediction). If there is sufficient trust in a community and aggregation system, I imagine this system could work decently, but if there isn’t, then it won’t.

In practice I would imagine that forecasters would be continually judged as future forecasts are contributed that agree or disagree with them, rather than only when definitive events happen that prove or disprove their forecasts. This means that forecasters could forecast things that happen in very long time horizons, and still be ranked based on their ability in the short term.

Going more abstract, there could be more abstract poll-like questions like, “How many soldiers died in war in WW2?” or “How many DALYs  would donating $10,000 to the AMF create in 2017?”. For these, individuals could propose their estimates, then the aggregation system would work roughly like normal to combine these estimates. Even though these questions may never be known definitively, if there is built in trust in the system, I could imagine that they could produce reasonable results.

One question here which is how to evaluate the results of aggregation systems for non-falsifiable questions. I don’t imagine any direct way, but could imagine ways of approximating it by asking experts how reasonable the results seem to them. While methods to aggregate results for non-falsifiable questions are themselves non-falsifiable, the alternatives also are very lacking. Given how many of these questions exist, it seems to me like perhaps they should be dealt with; and perhaps they can use the results from communities and statistical infrastructure optimized in situations that do have answers.


Each one of the above features could be described in much more detail, but I think the basic ideas are quite simple. I’m very enthusiastic about these, and would be interested in talking with anyone interested in collaborating on or just talking about similar tools. I’ve been considering attempting a system myself, but first want to get more feedback.


  1. The Good Judgement Project FAQ,

  2. Sharpening Your Forecasting Skills, Link

  3. IARPA Aggregative Contingent Estimation (ACE) research program

  4. The Good Judgement Project: A Large Scale Test of Different Methods of Combining Expert Predictions

  5. “Will Trump’s favorable rate be at least 45.0% on December 31st?” on PredictIt (Link).

  6. I believe Quantile Regression Averaging is one way of aggregating prediction intervals

  7. Hypermind (

Community needs, individual needs, and a model of adult development

12 Vaniver 17 December 2016 12:18AM

Sarah Constantin wrote:

Specifically, I think that LW declined from its peak by losing its top bloggers to new projects. Eliezer went to do AI research full-time at MIRI, Anna started running CFAR, various others started to work on those two organizations or others (I went to work at MetaMed). There was a sudden exodus of talent, which reduced posting frequency, and took the wind out of the sails.

One trend I dislike is that highly competent people invariably stop hanging out with the less-high-status, less-accomplished, often younger, members of their group. VIPs have a strong temptation to retreat to a "VIP island" -- which leaves everyone else short of role models and stars, and ultimately kills communities. (I'm genuinely not accusing anybody of nefarious behavior, I'm just noting a normal human pattern.) Like -- obviously it's not fair to reward competence with extra burdens, I'm not that much of a collectivist. But I think that potentially human group dynamics won't work without something like "community-spiritedness" -- there are benefits to having a community of hundreds or thousands, for instance, that you cannot accrue if you only give your time and attention to your ten best friends.

While I agree that the trend described in the second paragraph happens (and I also dislike the effects), I have another model that I think more tightly explains why the first paragraph happened. I also think that it's important to build systems with the constraint in mind that they work for the individuals inside those systems. A system that relies on trapped, guilted, or oppressed participants is a system at risk for collapse.

So in order to create great public spaces for rationalists, we don't just need to have good models of community development. Those can tell us what we need from people, but might not include how to make people fill those slots in a sustainable way.

To explain why lots of top bloggers left at once, let me present a model of adult development, drawn from George Vaillant’s modification of Erik Erikson’s model, as discussed in Triumphs of Experience. It focuses on 6 different ‘developmental tasks,’ rather than ‘stages’ or ‘levels.’ Each has success and failure conditions associated with it; a particular component of life goes either well or poorly. They’re also not explicitly hierarchical; one could achieve the “third” task before achieving the “second” task, for example, but one still notices trends in the ages at which the tasks have been completed.

Triumphs of Experience is the popsci treatment of the Harvard Grant Study of development; they took a bunch of Harvard freshmen and sophomores, subjected them to a bunch of psychological tests and interviews, and then watched them grow up over the course of ~70 years. This sort of longitudinal study gives them a very different perspective from cross-sectional studies, because they have much better pictures of what people looked like before and after.

I'll briefly list the developmental tasks, followed by quotes from Triumphs of Expertise that characterize succeeding at them. The bolded ones seem most relevant:


  • Identity vs. Role Diffusion: “Live independently of family of origin, and to be self-supporting.”
  • Intimacy vs. Isolation: “capacity to live with another person in an emotionally attached, interdependent, and committed relationship for ten years or more.”
  • Career Consolidation vs. Role Diffusion: “Commitment, compensation, contentment, and competence.”
  • Generativity vs. Stagnation: “assumption of sustained responsibility for the growth and well-being of others still young enough to need care but old enough to make their own decisions.”
  • Guardianship vs. Hoarding: The previous level covered one-on-one relationships; this involves more broad, future-focused sorts of endeavors. Instead of mentoring one person, one is caretaker of a library for many.
  • Integrity vs. Despair: Whether one is graceful in the face of death or not. [1]


As mentioned before, they're ordered but the ordering isn't strict, and so you can imagine someone working on any developmental task, or multiple at once. But it seems likely that people will focus most of their attention on their earliest ongoing task.

It seems to me like several of the top bloggers were focusing on blogging because something was blocking their attempts to focus on career consolidation, and so they focused on building up a community instead. When the community was ready enough, they switched to their career--but as people were all in the same community, this happened mostly at once.

I think that “community-spiritedness” in the sense that Sarah is pointing at, in the sense of wanting to take care of the raising of new generations or collection and dissemination of ideas, comes most naturally to people working on generativity and guardianship. People work on that most easily if they’re either done with consolidating their career or their career is community support (in one form or another). If not, it seems like there’s a risk that opportunities to pursue earlier needs will seem more immediate and be easily able to distract them.

(In retrospect, I fell prey to this; I first publicly embarked on the project of LW revitalization a year ago, and then after about a month started a search for a new job, which took up the time I had been spending on LW. If doing it again, I think the main thing I would have tried to do differently is budgeting a minimally sufficient amount of time for LW while doing the job search with the rest of my time, as opposed to spending time how it felt most useful. This might have kept the momentum going and allowed me to spend something like 80% of my effort on the search and 20% on LW, rather than unintentionally spending 100% and 0%.)


1. I interpret the last task through the lens of radical acceptance, not deathism; given that one is in fact dying, whether one responds in a way that helps themselves and those around them seems like an important thing to ask about separately from the question of how much effort we should put into building a world where that is the case later and less frequently than the present.

Combining Prediction Technologies to Help Moderate Discussions

12 Wei_Dai 08 December 2016 12:19AM

I came across a 2015 blog post by Vitalik Buterin that contains some ideas similar to Paul Christiano's recent Crowdsourcing moderation without sacrificing quality. The basic idea in both is that it would be nice to have a panel of trusted moderators carefully pore over every comment and decide on its quality, but since that is too expensive, we can instead use some tools to predict moderator decisions, and have the trusted moderators look at only a small subset of comments in order to calibrate the prediction tools. In Paul's proposal the prediction tool is machine learning (mainly using individual votes as features), and in Vitalik's proposal it's prediction markets where people bet on what the moderators would decide if they were to review each comment.

It seems worth thinking about how to combine the two proposals to get the best of both worlds. One fairly obvious idea is to let people both vote on comments as an expression of their own opinions, and also place bets about moderator decisions, and use ML to set baseline odds, which would reduce how much the forum would have to pay out to incentivize accurate prediction markets. The hoped for outcome is that the ML algorithm would make correct decisions most of the time, but people can bet against it when they see it making mistakes, and moderators would review comments that have the greatest disagreements between ML and people or between different bettors in general. Another part of Vitalik's proposal is that each commenter has to make an initial bet that moderators would decide that their comment is good. The article notes that such a bet can also be viewed as a refundable deposit. Such forced bets / refundable deposits would help solve a security problem with Paul's ML-based proposal.

Are there better ways to combine these prediction tools to help with forum moderation? Are there other prediction tools that can be used instead or in addition to these?

How I use Anki to learn mathematics

12 ArthurRainbow 07 December 2016 10:29PM
Here is my first less wrong post (after years spent blogging in French). I discovered  Anki on this blog. I'm now sharing the tips I've been using for months to learn mathematics with Anki. Example(s) of deck can be found on

I'm a French researcher in fundamental computer science, which essentially means that I do mathematics all day long. But my biggest problem is that I'm really bad at learning mathematics. Let me give an example. Hopefully, you should be able to understand even if you don't know the domains I'm using in my examples. One day, I wanted to discover what category theory is. Another day, I wanted to read an introduction to number theory. Two subjects which seem really interesting. And the first pages of those books seems to be crystal clear. After 10 or 20 pages, I have already forgotten the definitions given in the first page, and I am lost. Definitions are not hard, there is just too many of them.

Let me emphasize that I'm speaking of learning definitions you understand. As far as I know, it is mathematically useless to learn things you do not understand. (Apart, may be, if you try to  learn the first digits of Pi, or some few things like that).

In the category theory example, definitions are explained by showing how well-known properties of algebra or of topology are special cases of the introduced category definition. Those examples are really helpful to understand the definitions. Indeed, it allows my mind to go from new notions to known notions. For example, I recall that Epi and Mono are generalization of injective and of surjective. Or of surjective and injective. I can't remember which is which. But at least I know that an arrow can both be Epi and Mono. And even know that I know that Epi in Sets are surjective, I still don't know which of the properties of surjection remains. Which is a trouble since having only the Set example would imply that Epi and Mono imply Iso.

And, to solve this kind of problem, a spaced memorization software is wonderful. The only trouble I have is that the decks which already exist are not really interesting. Because, usually, card creation did not follow any special logic (some scanned hand written lectures note). And furthermore, the already existing decks concern mathematics I'm not interested in. I don't want to learn multiplication table, nor calculus. In fact, the only decks from the community I use are the one from Rationality, from A.I. to Zombie and the one which teach good learning practice. Hence, I'm now creating decks, from books on topic  I want to know.

The rules I follow


I create a deck per book. Hence I will be able to give the decks to the community, telling each time exactly what belongs in the deck. It means I accept to put in the decks things I already know. And to put the same thing in different decks (for example, the definition of Open Sets of course belongs to a topology deck. But it also belong to a complex analysis book where this definition is restricted to the case of metric space.) I don't think I'm really wasting my time, it allows me to learn things I don't perfectly know but that I would not have created otherwise. Furthermore, it will hopefully allows the community to begin by any book.

I must emphasize that the decks are supposed to help reading the book. Not to be used instead of the book. Because having read the proof at least once is often important. And my decks do not contains proofs (apart when I'm reading research paper and that my goal is to understand complex proofs).

I don't create the entire deck at once. For example, I'm reading a set theory book. Right now I've only read the first chapter. Because there are already too many definitions I do not remember correctly and which is confusing (the type, the cofinality, ...). Hence it  would be too much of a trouble to read the 2nd chapter yet. (Relating to this, I will need to tell the users of my decks to suspend the chapter n+1 until they know the content of the chapter n.)

Reciprocally, I create many decks simultaneously. Which is coherent with the way maths are studied in university. That is, even if I'm currently blocked in the complex analysis book, I can still read a graph theory book. And while I learn my analysis course, I can create the graph deck.

Kind of cards.

A basic anki card is a question and an answer. Sometime, both sides of the cards are questions and answer. It's useful for vocabulary. Since I'm French, I want to know both the English and the French name of the definitions I learn.

Otherwise, I only use clozed deletions. That is, a basic text, with holes you must recall. (Note that it allows to simulate basic cards. So once you selected cloze deletion, you never need to switch back to usual mode).

Apart from the usual «front» and «extra» field, I always have a «reference» field. In this field,  I write chapter, section, subsection and so on. I also write the theorem, lemma, corollary, definition number. I do not write down the page number because of laziness. I think I should, because some books have long sections without any numbered results. And in those case, it takes minutes to find where an information come from. Having theorem number is required because, in order to recall a result, it is usually helpful to remember its proof. And reciprocally, if you forgot the theorem, it may be usefull to read its proof again.

Last important fact. I told Anki not to erase fields when I completed the creation of a card. It is clear that chapter and section numbers rarely change, so it is useful not to have to write them down again. Concerning the mathematics part, I saw that successive cards are often similar. For example, in a linear algebra book, many results begin by «Let U,V be two vector space and T a morphism from u to V». It is helpful not to have to retype it in each cards.


In a definition card, there is usually three deletions. The first is the name of the defined object. The second is the notation of this object. The third one is the definition. If an object admits many equivalent definitions, all definitions appears on the same card. Each definition being a different cloze deletion. Indeed, if your card is «A square is ... » and you answer «A diamond with a 90° angle», you don't want to be wrong because it is written «A rectangle with two adjacent side of same length». Therefore, the card is:
«{{c1::A square}} is: equivalently
-{{c2::A diamond with a 90° angle}} or
-{{c3::A rectangle with two adjacent side of same length}}»

Beware, sometime it is better if the name and the notation belong to the same deletion. For example, if X is a subset of a vector space, it is easy to guess that «Aff(X)» is «the Affine Hull  of X» and that «the affine hull of X» is denoted «Aff(X)». What you really want is to recall that «the set of vectors of the form Sum of x_ir_i, with x_i\in X and r_i in the field, where the sum of the r_i is 1»  is «the affine hull of X».


A theorem usually admits two deletions. Hypothesis and conclusion. It sometime admits a third deletion if the theorem has a classical name. For example you may want to remember that the theorem stating «In a right triangle,  the square of the hypotenuse (the side opposite the right angle) is equal to the sum of the squares of the other two sides. » is called  «the Pythagorean theorem».

Beware, no hypothesis in a deletion should  introduce an object.  «If, ... , then the center of P is not trivial» is hard to understand, whereus «Let P be a group. If ..., then the center of P is non trivial» is more clear.


A first important thing is to always write all hypothesis. Sometime, some hypothesis are given in the beginning of a chapter and are assumed to be true in the whole chapter. But, when the card appears, you don't recall those hypothesis (and you don't want to learn that, in this very book, in this very chapter, only complex vector spaces are considered).

It is important to have empty deletion. In automata theory, some theorems deal with monoids, and some other only deal with finite monoid. If I write «If M is a {{c1::finite}} monoid» for theorem dealing with finite monoid, and  «If M is a monoid» for theorem dealing with arbitrary monoids, the deletion show that M is assumed finite. Therefore, in the second case, I must write «If M is a {{c1::}} monoid». It's really simple to do, since Anki don't delete the fields when I create a card. So, when the hypothesis that M is finite is not required anymore, I can keep the deletion in the field to create the new card.

Multiple implications

This leads me to another problem. An hypothesis may have many implications. And a property may be implied by many hypothesis. Therefore, it is better to state:

«If {{c1::P1}} or {{c2::P2}} .. or {{c3::Pi}} then {{c4::Q}}», or «If {{c1::P}} then {{c2::Q1}} and {{c3::Q2}} ... and {{c4::Qi}}». Indeed, you don't want to be wrong because, of all hypothesis, you thought of the wrong one for this very card. Here, each term in brackets represents a cloze deletion.

There would be a problem if P implies Q and R and if O and P  both imply Q. But this has not happened yet to me. It seems that it does not often happen in math textbooks.

Order of the words

When a theorem is of the form «{{c1::The  centralizer of A in G}} {{c2::divides}} {{c3::the normalizer of A in G}}», you must also have a card «The normalizer of A in G {{c1::is divided by}} the centralizer of  A in G». Otherwise, you could guess the result by pattern matching. Because you know the hole in the middle is either an equality or a dividability statement. Right now, I did not figure out how to do this efficiently. But I should edit my old cards in order to satisfy this property.

Note that you always want to have a cloze deletion named c1. If you don't have it, you can't preview your card, therefore you can't check whether your LaTeX compile. And if you choose to change c2 to c1 later, you need to review your card again, because Anki thinks that it is a new cloze deletion.


The last thing I want in Anki are examples. Generally, they are of the form 
«{{c1::A}}, {{c2::B}}.., {{c3::Z}} are examples of {{c4::P}}».

I used to believe that examples were not mathematics. Because, as a Bourbakist, as a logician, I know that an example does not belong to any theory. What matters is definitions, axioms, theorems, proofs. Since, I understood examples have at least three goals. Officially, it gives intuition, and shows cases where theorems can be applied. Formally, it shows that some set of hypothesis is not contradictory, hence it is useful to consider those hypothesis. In practice, this very examples may be used when one want to test statements which does not appear in the course. That is, examples are sometime answers to exercices, or counter example to idea one may have.


Once the cards are made, I ask to Anki to create all images using LaTex (which should be done immediatly in a perfect program, but is not). And I use synchronization to send all cards to my smartphone. (Free application, without ad, they are wonderful!) to learn in the public transport.

Of course, when I  create cards, I made mistake. Either because my LaTeX doesn't compile, and Anki shows an error message in the cards. Or because I forgot a word, wrote a word instead of another one. (For example, I always write «set» instead of «state».) If I believe there is a mistake, I suspend the card. Once at home, I synchronize and, on my computer, check whether it is really a mistake or not. 
It is why I wrote the chapter and section number of each card.  It allows me to check quickly whether the card and the book states the same thing.

Sometime, the mistake is a bad cloze deletion. It is possible that, when a part of the sentence is deleted, it makes no sens anymore. For example, it happens if I did not follow the above mentioned rules. (It was the case for the first deck I created, where I didn't devise any rule yet). Same rule applies if I see a mistake so big that I do not understand anymore the question I asked. 

Thanks to dutchie and to rhaps0dy who corrected many typo of this post.

Recent AI control posts

12 paulfchristiano 29 November 2016 06:53PM

Over at medium, I’m continuing to write about AI control; here’s a roundup from the last month.


  • Prosaic AI control argues that AI control research should first consider the case where AI involves no “unknown unknowns.”
  • Handling destructive technology tries to explain the upside of AI control, if we live in a universe where we eventually need to build a singleton anyway.
  • Hard-core subproblems explains a concept I find helpful for organizing research.

Building blocks of ALBA

Terminology and concepts

Using a Spreadsheet to Make Good Decisions: Five Examples

12 peter_hurford 28 November 2016 05:10PM

I've been told that LessWrong is coming back now, so I'm cross-posting this rationality post of interest from the Effective Altruism forum.


We all make decisions every day. Some of these decisions are pretty inconsequential, such as what to have for an afternoon snack. Some of these decisions are quite consequential, such as where to live or what to dedicate the next year of your life to. Finding a way to make these decisions better is important.

The folks at Charity Science Health and I have been using the same method to make many of our major decisions for the past for years -- everything from where to live to even deciding to create Charity Science Health. The method isn’t particularly novel, but we definitely think the method is quite underused.

Here it is, as a ten step process:

  1. Come up with a well-defined goal.

  2. Brainstorm many plausible solutions to achieve that goal.

  3. Create criteria through which you will evaluate those solutions.

  4. Create custom weights for the criteria.

  5. Quickly use intuition to prioritize the solutions on the criteria so far (e.g., high, medium, and low)

  6. Come up with research questions that would help you determine how well each solution fits the criteria

  7. Use the research questions to do shallow research into the top ideas (you can review more ideas depending on how long the research takes per idea, how important the decision is, and/or how confident you are in your intuitions)

  8. Use research to rerate and rerank the solutions

  9. Pick the top ideas worth testing and do deeper research or MVP testing, as is applicable

  10. Repeat steps 8 and 9 until sufficiently confident in a decision.


Which charity should I start?

The definitive example for this process was the Charity Entrepreneurship project, where our team decided which charity would be the best possible charity to create.

Come up with a well-defined goal: I want to start an effective global poverty charity, where effective is taken to mean a low cost per life saved comparable to current GiveWell top charities.

Brainstorm many plausible solutions to achieve that goal: For this, we decided to start by looking at the intervention level. Since there are thousands of potential interventions, we placed a lot of emphasis on plausibly highly effectve, and chose to look at GiveWell’s priority programs plus a few that we thought were worthy additions.

Create criteria through which you will evaluate those solutions / create custom weights for the criteria: For this decision, we spent a full month of our six month project thinking through the criteria. We weighted criteria based on both importance and the expected varaince that would occur between our options. We decided to strongly value cost-effectiveness, flexibility , and scalability. We moderately valued strength of evidence, metric focus, and indirect effects. We weakly valued logistical possibility and other factors.

Come up with research questions that would help you determine how well each solution fits the criteria: We came up with the following list of questions and research process.

Use the research questions to do shallow research into the top ideas, use research to rerate and rerank the solutions: Since this choice was important and we were pretty uninformed about the different interventions, we did shallow research into all of the choices. We then produced the following spreadsheet:

Afterwards, it was pretty easy to drop 22 out of the 30 possible choices and go with a top eight (the eight that ranked 7 or higher on our scale).


Pick the top ideas worth testing and do deeper research or MVP testing, as is applicable / Repeat steps 8 and 9 until sufficiently confident in a decision: We then researched the top eight more deeply, with a keen idea to turn them into concrete charity ideas rather than amorphous interventions. When re-ranking, we came up with a top five, and wrote up more detailed reports --SMS immunization reminders,tobacco taxation,iron and folic acid fortification,conditional cash transfers, and a poverty research organization. A key aspect to this narrowing was also talking to relevant experts, which we wish we did earlier on in the process as it could quickly eliminate unpromising options.

Pick the top ideas worth testing and do deeper research or MVP testing, as is applicable: As we researched further, it became more clear that SMS immunization reminders performed best on the criteria being highly cost-effective, with a high strength of evidence and easy testability. However, the other four finalists are also excellent opportunities and we strongly invite other teams to invest in creating charities in those four areas.


Which condo should I buy?

Come up with a well-defined goal: I want to buy a condo that is (a) a good place to live and (b) a reasonable investment.

Brainstorm many plausible solutions to achieve that goal: For this, I searched around on Zillow and found several candidate properties.

Create criteria through which you will evaluate those solutions: For this decision, I looked at the purchasing cost of the condo, the HOA fee, whether or not the condo had parking, the property tax, how much I could expect to rent the condo out, whether or not the condo had a balcony, whether or not the condo had a dishwasher, how bright the space was, how open the space was, how large the kitchen was, and Zillow’s projection of future home value.

Create custom weights for the criteria: For this decision, I wanted to turn things roughly into a personal dollar value, where I could calculate the benefits minus the costs. The costs were the purchasing cost of the condo turned into a monthly mortgage payment, plus the annual HOA fee, plus the property tax. The benefits were the expected annual rent plus half of Zillow’s expectation for how much the property would increase in value over the next year, to be a touch conservative. I also added some more arbitrary bonuses: +$500 bonus if there was a dishwasher, a +$500 bonus if there was a balcony, and up to +$1000 depending on how much I liked the size of the kitchen. I also added +$3600 if there was a parking space, since the space could be rented out to others as I did not have a car. Solutions would be graded on benefits minus costs model.

Quickly use intuition to prioritize the solutions on the criteria so far: Ranking the properties was pretty easy since it was very straightforward, I could skip to plugging in numbers directly from the property data and the photos.




Annual fees

Annual increase

Annual rent







































Come up with research questions that would help you determine how well each solution fits the criteria: For this, the research was just to go visit the property and confirm the assessments.

Use the research questions to do shallow research into the top ideas, use research to rerate and rerank the solutions: Pretty easy, not much changed as I went to actually investigate.

Pick the top ideas worth testing and do deeper research or MVP testing, as is applicable: For this, I just ended up purchasing the highest ranking condo, which was a mostly straightforward process. Property A wins! 
This is a good example of how easy it is to re-adapt the process and how you can weight criteria in nonlinear ways.

How should we fundraise? 

Come up with a well-defined goal: I want to find the fundraising method with the best return on investment. 

Brainstorm many plausible solutions to achieve that goal: For this, our Charity Science Outreach team conducted a literature review of fundraising methods and asked experts, creating a list of the 25 different fundraising ideas. 

Create criteria through which you will evaluate those solutions / Create custom weights for the criteria: The criteria we used here was pretty similar to the criteria we later used for picking a charity -- we valued ease of testing, the estimated return on investment, the strength of the evidence, and the scalability potential roughly equally. 

Come up with research questions that would help you determine how well each solution fits the criteria: We created this rubric with questions

  • What research says on it (e.g. expected fundraising ratios, success rates, necessary pre-requisites)

  • What are some relevant comparisons to similar fundraising approaches? How well do they work?

  • What types/sizes of organizations is this type of fundraising best for?

  • How common is this type of fundraising, in nonprofits generally and in similar nonprofits (global health)?

  • How one would run a minimum cost experiment in this area?

  • What is the expected time, cost, and outcome for the experiment?

  • What is the expected value?

  • What is the expected time cost to get best time per $ ratio (e.g., would we have to have 100 staff or huge budget to make this effective)?

  • What further research should be done if we were going to run this approach?

Use the research questions to do shallow research into the top ideas, use research to rerate and rerank the solutions: After reviewing, we were able to narrow the 25 down to eight finalists: legacy fundraising, online ads, door-to-door, niche marketing, events, networking, peer-to-peer fundraising, and grant writing.
Pick the top ideas worth testing and do deeper research or MVP testing, as is applicable: We did MVPs of all eight of the top ideas and eventually decided that three of the ideas were worth pursuing full-time: online ads, peer-to-peer fundraising, and legacy fundraising.

Who should we hire? 

Come up with a well-defined goal: I want to hire the employee who will contribute the most to our organization. 

Brainstorm many plausible solutions to achieve that goal: For this, we had the applicants who applied to our job ad.

Create criteria through which you will evaluate those solutions / Create custom weights for the criteria: We thought broadly about what good qualities a hire would have, and decided to heavily weight values fit and prior experience with the job, and then roughly equally value autonomy, communication skills, creative problem solving, the ability to break down tasks, and the ability to learn new skills.
Quickly use intuition to prioritize the solutions on the criteria so far: We started by ranking hires based on their resumes and written applications. (Note that to protect the anonymity of our applicants, the following information is fictional.)





Break down

Learn new skills

Values fit

Prior experience










































Come up with research questions that would help you determine how well each solution fits the criteria: The initial written application was already tailored toward this, but we designed a Skype interview to further rank our applicants. 

Use the research questions to do shallow research into the top ideas, use research to rerate and rerank the solutions: After our Skype interviews, we re-ranked all the applicants. 






Break down

Learn new skills

Values fit

Prior experience










































Pick the top ideas worth testing and do deeper research or MVP testing, as is applicable: While “MVP testing” may not be polite to extend to people, we do a form of MVP testing by only offering our applicants one month trials before converting to a permanent hire.


Which television show should we watch? 

Come up with a well-defined goal: Our friend group wants to watch a new TV show together that we’d enjoy the most. 

Brainstorm many plausible solutions to achieve that goal: We all each submitted one TV show, which created our solution pool. 

Create criteria through which you will evaluate those solutions / Create custom weights for the criteria: For this decision, the criteria was the enjoyment value of each participant, weighted equally. 

Come up with research questions that would help you determine how well each solution fits the criteria: For this, we watched the first episode of each television show and then all ranked each one. 

Pick the top ideas worth testing and do deeper research or MVP testing, as is applicable: We then watched the winning television show, which was Black Mirror. Fun! 


Which statistics course should I take? 

Come up with a well-defined goal: I want to learn as much statistics as fast as possible, without having the time to invest in taking every course. 

Brainstorm many plausible solutions to achieve that goal: For this, we searched around on the internet and found ten online classes and three books.

Create criteria through which you will evaluate those solutions / Create custom weights for the criteria: For this decision, we heavily weighted breadth and time cost, weighted depth and monetary cost, and weakly weighted how interesting the course was and whether the course provided a tangible credential that could go on a resume.
Quickly use intuition to prioritize the solutions on the criteria so far: By looking at the syllabi, table of contents, and reading around online, we came up with some initial rankings:



Estimated hours

Depth score

Breadth score

How interesting

Credential level

Master Statistics with R







Probability and Statistics, Statistical Learning, Statistical Reasoning







Critically Evaluate Social Science Research and Analyze Results Using R













Berkley stats 20 and 21







Statistical Reasoning for Public Health







Khan stats







Introduction to R for Data Science







Against All Odds







Hans Rosling doc on stats







Berkeley Math







OpenIntro Statistics







Discovering Statistics Using R by Andy Field







Naked-Statistics by Charles Wheelan








Come up with research questions that would help you determine how well each solution fits the criteria: For this, the best we could do would be to do a little bit from each of our top class choices, while avoiding purchasing the expensive ones unless free ones did not meet our criteria. 

Pick the top ideas worth testing and do deeper research or MVP testing, as is applicable: Only the first three felt deep enough. Only one of them was free, but we were luckily able to find a way to audit the two expensive classes. After a review of all three, we ended up going with “Master Statistics with R”.

[Link] Less costly signaling

12 paulfchristiano 22 November 2016 09:11PM

[Link] Crony Beliefs

12 ete 03 November 2016 08:54PM

Welcome to LessWrong (10th Thread, January 2017) (Thread A)

11 folkTheory 07 January 2017 05:43AM

(Thread B for January is here, created as a duplicate by accident)

Hi, do you read the LessWrong website, but haven't commented yet (or not very much)? Are you a bit scared of the harsh community, or do you feel that questions which are new and interesting for you could be old and boring for the older members?

This is the place for the new members to become courageous and ask what they wanted to ask. Or just to say hi.

The older members are strongly encouraged to be gentle and patient (or just skip the entire discussion if they can't).

Newbies, welcome!


The long version:


If you've recently joined the Less Wrong community, please leave a comment here and introduce yourself. We'd love to know who you are, what you're doing, what you value, how you came to identify as an aspiring rationalist or how you found us. You can skip right to that if you like; the rest of this post consists of a few things you might find helpful. More can be found at the FAQ.


A few notes about the site mechanics

To post your first comment, you must have carried out the e-mail confirmation: When you signed up to create your account, an e-mail was sent to the address you provided with a link that you need to follow to confirm your e-mail address. You must do this before you can post!

Less Wrong comments are threaded for easy following of multiple conversations. To respond to any comment, click the "Reply" link at the bottom of that comment's box. Within the comment box, links and formatting are achieved via Markdown syntax (you can click the "Help" link below the text box to bring up a primer).

You may have noticed that all the posts and comments on this site have buttons to vote them up or down, and all the users have "karma" scores which come from the sum of all their comments and posts. This immediate easy feedback mechanism helps keep arguments from turning into flamewars and helps make the best posts more visible; it's part of what makes discussions on Less Wrong look different from those anywhere else on the Internet.

However, it can feel really irritating to get downvoted, especially if one doesn't know why. It happens to all of us sometimes, and it's perfectly acceptable to ask for an explanation. (Sometimes it's the unwritten LW etiquette; we have different norms than other forums.) Take note when you're downvoted a lot on one topic, as it often means that several members of the community think you're missing an important point or making a mistake in reasoning— not just that they disagree with you! If you have any questions about karma or voting, please feel free to ask here.

Replies to your comments across the site, plus private messages from other users, will show up in your inbox. You can reach it via the little mail icon beneath your karma score on the upper right of most pages. When you have a new reply or message, it glows red. You can also click on any user's name to view all of their comments and posts.

All recent posts (from both Main and Discussion) are available here. At the same time, it's definitely worth your time commenting on old posts; veteran users look through the recent comments thread quite often (there's a separate recent comments thread for the Discussion section, for whatever reason), and a conversation begun anywhere will pick up contributors that way.  There's also a succession of open comment threads for discussion of anything remotely related to rationality.

Discussions on Less Wrong tend to end differently than in most other forums; a surprising number end when one participant changes their mind, or when multiple people clarify their views enough and reach agreement. More commonly, though, people will just stop when they've better identified their deeper disagreements, or simply "tap out" of a discussion that's stopped being productive. (Seriously, you can just write "I'm tapping out of this thread.") This is absolutely OK, and it's one good way to avoid the flamewars that plague many sites.

There's actually more than meets the eye here: look near the top of the page for the "WIKI", "DISCUSSION" and "SEQUENCES" links.
LW WIKI: This is our attempt to make searching by topic feasible, as well as to store information like common abbreviations and idioms. It's a good place to look if someone's speaking Greek to you.
LW DISCUSSION: This is a forum just like the top-level one, with two key differences: in the top-level forum, posts require the author to have 20 karma in order to publish, and any upvotes or downvotes on the post are multiplied by 10. Thus there's a lot more informal dialogue in the Discussion section, including some of the more fun conversations here.
SEQUENCES: A huge corpus of material mostly written by Eliezer Yudkowsky in his days of blogging at Overcoming Bias, before Less Wrong was started. Much of the discussion here will casually depend on or refer to ideas brought up in those posts, so reading them can really help with present discussions. Besides which, they're pretty engrossing in my opinion. They are also available in a book form.

A few notes about the community

If you've come to Less Wrong to  discuss a particular topic, this thread would be a great place to start the conversation. By commenting here, and checking the responses, you'll probably get a good read on what, if anything, has already been said here on that topic, what's widely understood and what you might still need to take some time explaining.

If your welcome comment starts a huge discussion, then please move to the next step and create a LW Discussion post to continue the conversation; we can fit many more welcomes onto each thread if fewer of them sprout 400+ comments. (To do this: click "Create new article" in the upper right corner next to your username, then write the article, then at the bottom take the menu "Post to" and change it from "Drafts" to "Less Wrong Discussion". Then click "Submit". When you edit a published post, clicking "Save and continue" does correctly update the post.)

If you want to write a post about a LW-relevant topic, awesome! I highly recommend you submit your first post to Less Wrong Discussion; don't worry, you can later promote it from there to the main page if it's well-received. (It's much better to get some feedback before every vote counts for 10 karma—honestly, you don't know what you don't know about the community norms here.)

Alternatively, if you're still unsure where to submit a post, whether to submit it at all, would like some feedback before submitting, or want to gauge interest, you can ask / provide your draft / summarize your submission in the latest open comment thread. In fact, Open Threads are intended for anything 'worth saying, but not worth its own post', so please do dive in! Informally, there is also the unofficial Less Wrong IRC chat room, and you might also like to take a look at some of the other regular special threads; they're a great way to get involved with the community!

If you'd like to connect with other LWers in real life, we have  meetups  in various parts of the world. Check the wiki page for places with regular meetups, or the upcoming (irregular) meetups page. There's also a Facebook group. If you have your own blog or other online presence, please feel free to link it.

If English is not your first language, don't let that make you afraid to post or comment. You can get English help on Discussion- or Main-level posts by sending a PM to one of the following users (use the "send message" link on the upper right of their user page). Either put the text of the post in the PM, or just say that you'd like English help and you'll get a response with an email address. 
Barry Cotter

A note for theists: you will find the Less Wrong community to be predominantly atheist, though not completely so, and most of us are genuinely respectful of religious people who keep the usual community norms. It's worth saying that we might think religion is off-topic in some places where you think it's on-topic, so be thoughtful about where and how you start explicitly talking about it; some of us are happy to talk about religion, some of us aren't interested. Bear in mind that many of us really, truly have given full consideration to theistic claims and found them to be false, so starting with the most common arguments is pretty likely just to annoy people. Anyhow, it's absolutely OK to mention that you're religious in your welcome post and to invite a discussion there.

A list of some posts that are pretty awesome

I recommend the major sequences to everybody, but I realize how daunting they look at first. So for purposes of immediate gratification, the following posts are particularly interesting/illuminating/provocative and don't require any previous reading:

More suggestions are welcome! Or just check out the top-rated posts from the history of Less Wrong. Most posts at +50 or more are well worth your time.

Welcome to Less Wrong, and we look forward to hearing from you throughout the site!

Buckets and memetic immune disorders

11 Tyrrell_McAllister 03 January 2017 11:51PM

AnnaSalamon's recent post on "flinching" and "buckets" nicely complements PhilGoetz's 2009 post Reason as memetic immune disorder. (I'll be assuming that readers have read Anna's post, but not necessarily Phil's.) Using Anna's terminology, I take Phil to be talking about the dangers of merging buckets that started out as separate. Anna, on the other hand, is talking about how to deal with one bucket that should actually be several.

Phil argued (paraphrasing) that rationality can be dangerous because it leads to beliefs of the form "P implies Q". If you convince yourself of that implication, and you believe P, then you are compelled to believe Q. This is dangerous because your thinking about P might be infected by a bad meme. Now rationality has opened the way for this bad meme to infect your thinking about Q, too.

It's even worse if you reason yourself all the way to believing "P if and only if Q". Now any corruption in your thinking about either one of P and Q will corrupt your thinking about the other. In terms of buckets: If you put "Yes" in the P bucket, you must put "Yes" in the Q bucket, and vice versa. In other words, the P bucket and the Q bucket are now effectively one and the same.

In this sense, Phil was pointing out that rationality merges buckets. (More precisely, rationality creates dependencies among buckets. In the extreme case, buckets become effectively identical). This can be bad for the reasons that Anna gives. Phil argues that some people resist rationality because their "memetic immune system" realizes that rational thinking might merge buckets inappropriately. To avoid this danger, people often operate on the principle that it's suspect even to consider merging buckets from different domains (e.g., religious scripture and personal life).

This suggests a way in which Anna's post works at the meta-level, too.

Phil's argument is that people resist rationality because, in effect, they've identified the two buckets "Think rationally" and "Spread memetic infections". They fear that saying "Yes" to "Think rationally" forces them to say "Yes" to the dangers inherent to merged buckets.

But Anna gives techniques for "de-merging" buckets in general if it turns out that some buckets were inappropriately merged, or if one bucket should have been several in the first place.

In other words, Anna's post essentially de-merges the two particular buckets "Think rationally" and "Spread memetic infections". You can go ahead and use rational thinking, even though you will risk inappropriately merging buckets, because you now have techniques for de-merging those buckets if you need to.

In this way, Anna's post may diminish the "memetic immune system" obstacle to rational thinking that Phil observed.

Feature Wish List for LessWrong

11 sarahconstantin 17 December 2016 09:10PM

Efforts are underway to replace the old LessWrong codebase. This is a thread to solicit people's ideas and requests for features or changes to the LessWrong website.  What would make discussion quality better?

Matching donation fundraisers can be harmfully dishonest.

11 Benquo 11 November 2016 09:05PM

Anna Salamon, executive director of CFAR (named with permission), recently wrote to me asking for my thoughts on fundraisers using matching donations. (Anna, together with co-writer Steve Rayhawk, has previously written on community norms that promote truth over falsehood.) My response made some general points that I wish were more widely understood:

  • Pitching matching donations as leverage (e.g. "double your impact") misrepresents the situation by overassigning credit for funds raised.
  • This sort of dishonesty isn't just bad for your soul, but can actually harm the larger world - not just by eroding trust, but by causing people to misallocate their charity budgets.
  • "Best practices" for a charity tend to promote this kind of dishonesty, because they're precisely those practices that work no matter what your charity is doing.
  • If your charity is impact-oriented - if you care about outcomes rather than institutional success - then you should be able to do substantially better than "best practices".

So I'm putting an edited version of my response here.

continue reading »

*How* people shut down thought because of high-status respectable halos

11 NancyLebovitz 20 October 2016 02:09PM

A detailed look at the belief that high status social structures can be so much better than anything one can think of that there's no point in even trying to think about the details of what to do, and how debilitating this is.

Discussion of the essay

View more: Next