See also: Reflections on rationality a year out
My favorite part of Lord of the Rings was skipped in both film adaptations. It occurs when our four hobbit heroes (Sam, Frodo, Merry and Pippin) return to the Shire and learn it has been taken over by a gang of ruffians. Merry assumes Gandalf will help them free their home, but Gandalf declines:
I am not coming to the Shire. You must settle its affairs yourselves; that is what you have been trained for... My dear friends, you will need no help. You are grown up now. Grown indeed very high...
As it turns out, the hobbits have acquired many powers along their journey — powers they use to lead a resistance and free the Shire.
That is how I felt when I flew home for the holidays this December. Minnesota wasn't ruled by ruffians, but the familiar faces and places reminded me of the person I had been before I moved away, just a few years ago.
And I'm just so much more powerful than I used to be.
And in my case, at least, many of my newfound powers seem to come from having seriously leveled up in rationality.
Power 0: Curiosity
I was always "curious," by which I mean I felt like I wanted to know things. I read lots of books and asked lots of questions. But I didn't really want to know the truth, because I didn't care enough about the truth to study, say, probability theory and the cognitive science of how we deceive ourselves. I just studied different Christian theologies — and, when I was really daring, different supernatural religions — and told myself that was what honest truth-seeking looked like.
It took 20 years for reality to pierce my comfortable, carefully cultivated bubble of Christian indoctrination. But when it finally popped, I realized I had (mostly) wasted my life thus far, and I was angry. Now I studied things not just for the pleasure of discovery and the gratifying feeling of caring about truth, but because I really wanted an accurate model of the world so I wouldn't do stupid things like waste two decades of life.
And it was this curiosity, more than anything else, that led to everything else. So long as I burned for reality, I was bound to level up.
Power 1: Belief Propagation
One factor that helped religion cling to me for so long was my ability to compartmentalize, to shield certain parts of my beliefs from attack, to apply different standards to different beliefs like the scientist outside the laboratory. When genuine curiosity tore down those walls, it didn't take long for the implications of my atheism to propagate. I noticed that contra-causal free will made no sense for the same reasons God made no sense. I noticed that whatever value existed in the universe was made of atoms. I assumed the basics of transhumanism without knowing there was a thing called "transhumanism." I noticed that minds didn't need to be made of meat, and that machines could be made more moral than humans. (I called them "artificial superbrains" at the time.) I noticed that scientific progress could actually be bad, because it's easier to destroy the world than to protect it. I also noticed we should therefore "encourage scientific research that saves and protects lives, and discourage scientific research that may destroy us" — and this was before I had read about existential risk and "differential technological development."
Somehow, I didn't notice that naturalism + scientific progress also implied intelligence explosion. I had to read that one. But when I did, it set off another round of rapid belief updates. I noticed that the entire world could be lost, that moral theory was an urgent engineering problem, that technological utopia is actually possible (however unlikely), and more.
The power of belief propagation gives me clarity of thought and coherence of action. My actions are now less likely to be informed by multiple incompatible beliefs, though this still occurs sometimes due to cached thoughts.
Power 2: Scholarship
I was always one to look things up, but before my deconversion my scholarship heuristic seems to have been "Find something that shares most of my assumptions and tells me roughly what I want to hear, filled with lots of evidence to reassure me of my opinion." That's not what I thought I was doing at the time, but looking back at my reading choices, that's what it looks like I was doing.
After being taken by genuine curiosity, my heuristic became something more like "Check what the mainstream scientific consensus is on the subject, along with the major alternative views and most common criticisms." Later, I added qualifications like "But watch out for signs that an entire field of inquiry is fundamentally unsound."
The power of looking shit up proved to have enormous practical value. How could I make Common Sense Atheism popular, quickly? I studied how to build blog traffic, applied the major lessons, and within 6 months I had one of the most popular atheism blogs on the internet. How could I improve my success with women? I skim-read dozens of books on the subject, filtered out the best advice, applied it (after much trepidation), and eventually had enough success that I didn't need to worry about it anymore. What are values, and how do they work? My search lead me from philosophy to affective neuroscience and finally to neuroeconomics, where I hit the jackpot and wrote A Crash Course in the Neuroscience of Human Motivation. How could I be happier? I studied the science of happiness, applied its lessons, and went from occasionally suicidal to stably happy. How could I make the Singularity Institute more effective? I studied non-profit management and fundraising, and am currently (with lots of help) doing quite a lot to make the organization more efficient and credible.
My most useful scholarship win had to do with beating akrasia. Eliezer wrote a post about procrastination that drew from personal anecdote but not a single experiment. This prompted me to write my first post, which suggested he ought to have done a bit of research on procrastination, so he could stand on the shoulders of giants. A simple Google scholar search on "procrastination" turned up a recent "meta-analytic and theoretical review" of the field as the 8th result, which pointed me to the resources I used to write How to Beat Procrastination. Mastering that post's algorithm for beating akrasia might be the most useful thing I've ever done, since it empowers everything else I try to do.
Power 3: Acting on Ideas
Another lesson from my religious deconversion was that abstract ideas have consequences. Because of my belief in the supernatural, I had spent 20 years (1) studying theology instead of math and science, (2) avoiding sexual relationships, and (3) training myself in fantasy-world "skills" like prayer and "sensing the Holy Spirit." If I wanted to benefit from having a more accurate model of the world as much as I had been harmed by having a false model, I'd need to actually act in response to the most probable models of the world I could construct.
Thus, when I realized I didn't like the Minnesota cold and could be happy without seeing my friends and family that often, I threw all my belongings in my car and moved to California. When I came to take intelligence explosion seriously, I quit my job in L.A., moved to Berkeley, interned with the Singularity Institute, worked hard, got hired as a researcher, and was later appointed Executive Director.
Winning with Rationality
These are just a few of my rationality-powers. Yes, I could have gotten these powers another way, but in my case they seemed to flow largely from that first virtue of rationality: genuine curiosity. Yes, I've compressed my story and made it sound less messy than it really was, but I do believe I've been gaining in rationalist power — the power of agency, systematized winning — and that my life is much better as a result. And yes, most people won't get these results, due to things like akrasia, but maybe if we figure out how to teach the unteachable, those chains won't hold us anymore.
What does a Level 60 rationalist look like? Maybe Eliezer Yudkowsky + Tim Ferris? That sounds like a worthy goal! A few dozen people that powerful might be able to, like, save the world or something.
I'm concerned with the overuse of the term "applause light" here.
An applause light is not as simple as "any statement that pleases an in-group". The way I read it, a charge of applause lights requires all of the following to hold:
1) There are no supporting details to provide the statement with any substance.
2) The statement is a semantic stopsign.
3) The statement exists purely to curry favor with an in-group.
4) No policy recommendations follow from that statement.
I don't see a bunch of applause lights when I read this post. I see a post overflowing with supporting details, policy recommendations, and the opposite of semantic stopsigns -- Luke actually bent over backwards and went to the trouble of linking to as many useful posts on the topic as he could find. By doing so, he's giving the curious reader a number of pointers to what others have said about the subject he's discussing -- so that they can go learn more if they're actually curious.
Really, what more could he have done? How was he supposed to discuss the massive utility he's gained from rationality without mentioning rationality? To make his post shorter, Luke had to use several terms that most people around here feel good about. Yay for Luke! He saved me from having to read a longer, less information-dense post by writing it this way. I understand the sanity benefits of guarding yourself against blatant applause lights, but at the same time, it would be rather perverse of me to automatically feel unhappy in response to Luke mentioning something that makes me happy.
It's not an affective death spiral for me to feel happy when someone tells me an inspiring life-success story that involves terms that I happen to have a positive affect for. It's having a reaction that fits the facts. I'm happy Luke is having a good life. It's relevant for him to tell me about it here on Less Wrong because rationality played a big part in his success. And I'm even more overjoyed and grateful to Luke that he's leaving a trail of sign-posts behind him, pointing the way forward as he levels up in rationality. Now is the time for him to be documenting this... while it's still fresh... so that one day when he forgets how he got to where he is, there will still be an overly detailed record to point people to.
I agree. Whatever the reason is for me being annoyed or uncomfortable about lukeprog's writing on occasion is probably not because it is Applause Lights. It may even be that he is following Best Practices and I should just adjust to it.
On the other hand, I think I get the feeling that when someone links to something external, they are trying to activate a cognitive shortcut. It's like being fast-talked by a car salesman, I'm being given a reference instead of a whole concept. I'm afraid that I will accept something without actually forming a complete menta... (read more)