I was young, and a mere Traditional Rationalist who knew not the teachings of Tversky and Kahneman.
He means Judgment under Uncertainty: Heuristics and Biases, almost certainly. I think at one point there was a reading group surrounding it, but I don't know what ever happened to them.
Today, one of the chief pieces of advice I give to aspiring young rationalists is "Do not attempt long chains of reasoning or complicated plans."
Advice more or less completely ignored by everyone, including EY himself.
To a Bayesian, on the other hand, if a hypothesis does not today have a favorable likelihood ratio over "I don't know", it raises the question of why you today believe anything more complicated than "I don't know". But I knew not the Way of Bayes, so I was not thinking about likelihood ratios or focusing probability density.
I want to point out that thinking about likelihood ratios or focusing probability density is independent of any knowledge of Bayes' theorem. I'd be surprised if any calculation actually occurred.
When I think about how my younger self very carefully followed the rules of Traditional Rationality in the course of getting the answer wrong, it sheds light on the question of why people who call themselves "rationalists" do not rule the world. You need one whole hell of a lot of rationality before it does anything but lead you into new and interesting mistakes.
I don't understand this argument. What does calling yourself rationalist have to do with not ruling the world? Traditional rationality has all sorts of specialized counter-memes against political activity. Hell, even LW has counter-memes against entering politics. Isn't it plausible that traditional rationalists eschew politics for these reasons, rather than EY's thesis, which always seems to be along the lines of "Traditional Rationality is good for nothing, or perhaps less than nothing."
Traditional Rationality is taught as an art, rather than a science; you read the biography of famous physicists describing the lessons life taught them, and you try to do what they tell you to do. But you haven't lived their lives, and half of what they're trying to describe is an instinct that has been trained into them.
The crushing irony of reading an autobiographical anecdote of EY's in an essay that implicitly says "learn this intuition I stumbled across" is almost too great to bear. Certainly LW-style rationality is no more a science than traditional rationality is.
The way Traditional Rationality is designed, it would have been acceptable for me to spend 30 years on my silly idea, so long as I succeeded in falsifying it eventually, and was honest with myself about what my theory predicted, and accepted the disproof when it arrived, et cetera.
I don't think there's enough data presented here to actually support this claim, depending upon what EY means by "acceptable." Would the community support him financially if he spent 30 years trying to demonstrate quantum consciousness, without producing something intrinsically valuable along the way? I don't think so; even Penrose had to produce sound mathematics to back up his crackpot-ness.
Traditional Rationalists can agree to disagree.
The LW advance in this area seems to consist entirely of agreeing to disagree after quoting Aumann's theorem and continuing the argument well past the point of diminishing returns. In that respect, Traditional Rationalists win, merely because they don't have to put up with as much bullshit from e.g., religious nutjobs playing at rationality.
Maybe that will be enough to cross the stratospherically high threshold required for a discipline that lets you actually get it right, instead of just constraining you into interesting new mistakes.
If modern LW is any indication, then it probably wasn't enough. Everyone talks about Bayes, but few people do any actual math. EY wrote a whole sequence on quantum physics, writing the Schrodinger operator exactly once. If math is what will save us from making interesting new mistakes, we clearly aren't doing enough of it.
I think 'do not rule the world' meant something 'are not highly influential in the world, being CEOs, influential politicians, directors or large scientific projects, etc'.
Today's post, My Wild and Reckless Youth was originally published on 30 August 2007. A summary (taken from the LW wiki):
Discuss the post here (rather than in the comments to the original post).
This post is part of the Rerunning the Sequences series, where we'll be going through Eliezer Yudkowsky's old posts in order so that people who are interested can (re-)read and discuss them. The previous post was Say Not "Complexity", and you can use the sequence_reruns tag or rss feed to follow the rest of the series.
Sequence reruns are a community-driven effort. You can participate by re-reading the sequence post, discussing it here, posting the next day's sequence reruns post, or summarizing forthcoming articles on the wiki. Go here for more details, or to have meta discussions about the Rerunning the Sequences series.