I think the most important idea of the Sequences has aged well: the idea that you should have multiple working hypotheses, instead of falling in love with only one. It's summarized in T.C. Chamberlin's paper and Abram's post.

But Eliezer's sweet coating around it, the reporting on scientific controversies, hasn't aged so well:

• The replication crisis has hit many studies that Kahneman relied on

• Neural networks have outpaced neat AI

• Statisticians no longer spend much time arguing Bayesianism vs frequentism

• Physicists no longer spend much time arguing many-worlds vs collapse

Due to that, anyone trying to learn from the Sequences today will develop a certain archaic slant that isn't shared by experts. Getting rid of it took me some effort and embarrassment. So if we want an updated textbook, let's start by reporting on today's actual scientific controversies. (We should keep the CDT vs EDT controversy though, it has worked out well for us. And of course past controversies like religion or phlogiston should stay too.)

I discussed a few of the points here with some people at the MIRI lunch table, and Scott Garrabrant pointed out "hey, I loudly abandoned Bayesianism!". That is, we always knew that the ideal Bayesianism required infinite computation (you don't just consider a few hypotheses, but all possible hypotheses) and wouldn't work for embedded agents, and as MIRI became more interested in embedded agency they started developing theories of how that works. There was some discussion of how much this aligned with various people's claim that the quantitative side of Bayes wasn't all that practical for humans (with, I think, the end result being seeing them as similar).

For example, in August 2013 there was some discussion of Chapman's Pop Bayesianism, where I said:

I think the actual quantitative use of Bayes is not that important for most people. I think the qualitative use of Bayes is very important, but is hard to discuss, and I don’t think anyone on LW has found a really good way to do that yet. (CFAR is trying, but hasn’t come up with anything spectacular so far to the best of my knowledge.)

Then Scott Alexander responded, identifying Bayesianism in contrast to other epistemologies, and I identified some qualitative things I learned from Bayes, as did Tyrrell McAllister.

How does this hold up, five years later?

I still think Bayesianism as synthesis of Aristotelianism and Anton-Wilsonism is superior to both; I think the operation underlying Bayesianism for embedded agents is not Bayesian updating, but rather something that approaches Bayesian updating in the limit, and that one of the current areas of progress in rationality is grappling with what's actually going there. (Basically, this is because of the standard Critical Rationalist critique of Bayesianism, that the Bayesian view says the equivalent of "well, you just throw infinite compute at the problem to consider the superset of all possible answers, and then you're good" which is not useful advice to current practicing scientists. But the CR answer doesn't appear to be good enough either.)

I think basically the same things I did then--that the actual quantitative use of Bayes is not that important for most people, and that CFAR's techniques for talking about the qualitative use of Bayes mostly don't refer to Bayes directly. I don't think this state of affairs represents a 'school without practitioners' / I still disagree with The Last Rationalist's assessment of things, but perhaps I'm missing what TLR is trying to point at.

I'm a CFAR instructor, but the following views are my personal opinion.

All of this is to support a claim that should be fairly obvious, but I suspect many readers will try to wriggle out of if I state it without extensive justification: Bayesian methods are a core feature of Eliezer Yudkowsky's version of rationality. You might even say that Eliezer's variant could be called "Bayesian Rationality". It's not a 'technique' or a 'tool', to Eliezer Bayes is the law, the irrefutable standard that provides a precise unchanging figure for exactly how much you should update in response to a new piece of evidence. Bayes shows you that there is in fact a right answer to this question, and you're almost certainly getting it wrong.

This in turn points toward the uncomfortable fact that Bayes does not seem to have helped the Bayesian Rationalists develop useful approximations of correct inference. In fact, it's not so much that we started with primitive approximations and then improved them. Rather, the Bayesian feature of Eliezer's philosophy seems to have left no conceptual descendants in the meme pool. For example, the Center For Applied Rationality's 2017 handbook does not include the phrase "Bayes Theorem" even once. It's taken by the current cohort as something of a status symbol, a neat novelty you can claim to have knowledge of to boost prestige.

I think section is basically wrong, both about how Bayes relates to Eliezer's overall philosophy and how it's impacted the rationality community. On Eliezer's philosophy, take this bit from Twelve Virtues of Rationality:

You may try to name the highest principle with names such as “the map that reflects the territory” or “experience of success and failure” or “Bayesian decision theory”. But perhaps you describe incorrectly the nameless virtue. How will you discover your mistake? Not by comparing your description to itself, but by comparing it to that which you did not name.

I find it harder to assess how the rationality community or its meme pool have been affected, mostly because I find it hard to separate out what I know and what the community knows. I was essentially a professional statistician before joining the rationality community and so was well familiar with Bayes and probabilistic reasoning; the main thing I got from Eliezer's form of Bayesianism was applying it to philosophical positions and intuitions and similar things, where I claim the primary thing of importance is the basic conceptual structure, rather than the actual math.

Which gets to the question of CFAR classes and conceptual descendants. CFAR did teach classes on Bayes, back in the old days, and found that they didn't help students--either the students already knew, or they didn't get much out of it. But whether or not current classes are conceptual descendants of Bayes presumably depends on whether the concepts are related, not whether or not Bayes is referenced. Indeed,if Bayes Theorem were mostly a status symbol, wouldn't it be surprising for it to not be present in the handbook, just as an applause light?

Taking Double Crux as an example of a central CFAR technique, I claim the core feature that distinguishes double crux from other methods of communication is that it focuses on cruxes, that is, subbeliefs that would change your mind on the superbelief, or in Bayesian terms, P(A|B) is high and P(A|~B) is low, rather than justifications, that is, arguments that establish that P(A) is high. The 'double' bit means that both of us disagree on the other fact B, and both of us think it's meaningful. This means we're attempting to jointly optimize our expected Bayesian update, essentially, rather than trying to convince each other, in a way that makes it clearly a conceptual descendant of Bayes Theorem. Now, it's slightly awkward that the standard presentation of double crux treats beliefs as logical instead of probabilistic, but this seems to me to be a pedagogical tool instead of a conceptual flaw. (Indeed, the Bayesian treatment of double crux is obvious and smoothly connects to the practice of double crux in the wild.)

From the OP:

This is fairly close to the situation we find ourselves in with the bias literature. But nobody seems particularly shaken, and why should they? Our naive impression is just that, naivete. The straightforward conclusion is that if deleting knowledge from the canon causes no reaction, then it clearly wasn't important to people.

From one of Eliezer's recent posts:

(I’ve read a number of replications and variations on this research, and the effect size is blatant. I would not expect this to be one of the results that dies to the replication crisis, and I haven’t yet heard about the replication crisis touching it. But we have to put a maybe-not marker on everything now.)

From the OP again:

In my next post I'll show how this dynamic came about, and what it looks like when a community actively updates its knowledge in response to new information and events.

This seems potentially interesting, but I am somewhat worried that the model won't break down 'community' at a sufficient level of detail that it actually ends up being useful, or (more to my interests) won't engage with the tech of LW2.0. Curation, sequences, and similar things are attempts to make it such that new users see not just R:AZ but also Scott Alexander's sequences and also Inadequate Equilibria and so on. That said, a community is much more than its new members, and I suspect dealing with the ecosystem of individuals involved requires a wide view and many tools. [I am also not terribly impressed with the other posts on the blog so far, but I hope those to come will be useful.]

So how many "confirmed kills" of ideas found in the sequences actually are there? I know the priming studies got eviscerated, but the last time I looked into this I couldn't exactly find an easy list of "famous psychology studies that didn't replicate" to compare against.