Yes, we should definitely make CFAR/LW look more like a cult :-/

[CFAR's newest instructor, here; longtime educator and transhumanist-in-theory with practical confusions]

ScottL—I'm just coming out of the third workshop in six weeks, and flying to Boston to give some talks, so I'm exhausted and haven't had a chance to read through your compilation yet. I will, soon (+1 for the effort you've put forth), but in the meantime I wanted to pop in and give some thoughts on the comments thus far.

Benito, Rainbow, and Crux—+1 for all three perspectives.

Can CFAR content be learned from a compilation or writeup? Yes. After all, it's not magic—it was developed by careful thinkers looking at research and at their own cognition, iterated over 20+ formal attempts (and literally hundreds of informal ones) to share those same insights with others. It's complex, but it's also fundamentally discoverable.

However, there are three large problems (as I see it, speaking as the least experienced staff member). The first is the most obvious—it's hard. It's hard like learning karate from text descriptions is hard. If you go about this properly, without being sloppy or taking shortcuts or making dangerous assumptions, then you're in for a LONG, difficult haul. Speaking as someone who pieced together the discipline of parkour back in 2003, from scattered terrible videos (pre Youtube) and a few internet comment boards—pulling together a cohesive and working practice from even the best writeups is a tremendously difficult task. It's better on almost every axis with instructors, mentors, friends, companions—people to help you avoid the biggest pitfalls, help you understand the subtle points, tease apart the interesting implications, shore up your motivation, assist you in seeing your own mistakes and weaknesses. None of that is impossible on your own, but it's somewhere between one and two orders of magnitude more efficient and more efficacious with guidance.

The second is corruption. As Benito points out, a large part of the problem of rationality instruction is finding things that actually work—if mere knowledge of the flaws were sufficient to protect us from the flaws, then everybody who cared enough could just slog through Heuristics and Biases and be something like 70% of the way there. We've already put several thousand thought-hours and 20+ iterations into tinkering with content, scaffolding, presentation, and practice. What we've got works pretty well, but progress has been incremental and cumulative. What we had before worked less well, and what we had before that worked less well still.

Picture throwing out a complete text version of our current best practices, exposing it to the forces of memetic selection and evolution. Fragments would get seized upon, and quoted out of context; bits of it would get mixed up with this and that; things would be presented out of order and read out of order; people would skip and skim and possibly completely ignore sections they THOUGHT they already knew because the title or the first paragraph seemed mundane or familiar. And there wouldn't be the strong selection pressure toward clarity and cohesion that we've been providing, top-down—instead, there would be selection pressures for what's memorable, pithy, or easily crystallized, none of which would be likely to drive the art forward and make the content BETTER. Each step away from our current best practices is much more likely to be a decrease in quality rather than an increase, and though you and others here on LW are likely to have the necessary curiosity and diligence to "do it right," that doesn't mean that the majority of people exposed to the memes in this way share your autodidactic rigor.

The third problem (related to the second) is idea inoculation. Having seen crappy, distorted versions of the CFAR curriculum (or having attempted to absorb it from text, and failed), a typical human would then be much, much less receptive to other, better explanations in the future. This is why, even within the context of the workshop, we often ask that participants not read the relevant sections of their workbooks until AFTER a given lecture or activity. I'm going to assume this is a familiar concept, and not spend too many words on it, but suffice it to say that I believe an uncanny valley version of our curriculum trending on the internet for one day could produce enough anti-rationality in the general population to counterbalance all of our efforts so far.

None of these problems are absolute in nature. The Sequences exist, and are known to be helpful. And clearly, Rainbow and Benito have gotten at least some value out of the writeups they've gleaned and assembled themselves. Again, there's nothing to stop others from having the same insights we've had, and there's nothing to stop a diligent autodidact from connecting scattered dots.

But they are statistical. They are real. They become quite scary, once you start talking big numbers of people and the free exchange of content-sans-context. And that's without even talking about other concerns like framing, signaling, inferential distance, etc. Lots of worms in this can.

So the question then becomes—what to do?

Thus far, CFAR hasn't had the cycles to spend time creating the (let's say) 80-20 version of their content. Remember that it's a fledgling startup with fewer than ten full-time staff members (when Pete and I were hired, it only had six). They were pouring every 60- and 70- and 80-hour week into trying to squeeze an extra percentage point of comprehension or efficacy out of every activity, every explanation. In other words, the objection wasn't fundamental (to the best of my understanding, which may be wrong) ... it was pragmatic. Creating packaged material fit for the general public wasn't anywhere near the top of the list, which was headed by "create material that's actually epistemically sound and demonstrably effective."

For my own part, I think this belongs in our near future. I think it's an area to be approached cautiously, in incremental steps with lots of data collection, but yes—I'd like to see some of our simpler, core techniques made broadly available. I'd like to see scalability in the things we think we can actually explain on paper. And if it goes well, I'd like to see more and more of it. I'm personally taking steps in this direction (tackling and improving our written content is one of my primary tasks, and I've started with simple things like drafting a glossary and tracking which definitions leave the reader confused (or worse, confident but wrong)).

But we have to a) find the time and manpower to actually run the experiment, and b) find content that genuinely works. Those are both non-trivially difficult, and they're both trading off against the continued expansion and improvement of our version of the art of rationality. I've only just now taken on enough responsibility myself to free up a few of the core staff's hours—and that's mostly gone into reducing their workload from insane to merely crazy. It hasn't actually created sufficient surplus to allow online tutorials to meet the threshold for worth-the-risks.

In short, despite Crux's entirely appropriate and reasonable skepticism, the answer has to be (for the immediate future)—either you find us trustworthy, or you don't (and if you don't, maybe you don't want our material anyway?). I, for one, don't think published material threatens workshop revenue, any more than online tutorials threaten martial arts dojos. There will always be obvious benefits to attending an intensive, collaborative workshop with instructors who know what they're doing, and there will always be people who recognize that the value is worth the cost, particularly given our track record. Our reasons for having refrained from publication thus far aren't monetary (or, to be more precise, money isn't in the top five on what's actually a fairly long and considered list).

Instead, it's that we actually care about getting it right. We don't want to poison the well, we don't want to break the very thing we're trying to protect, and as a member of a group with something that at least resembles expertise (if you don't want to credit us as actual experts), I think that requires a lot more work on our end, first.

That being said, if you have questions about the content above, or about what CFAR is doing this week and this month and this year, or if you're struggling with creating the art of rationality yourself and you've had novel and interesting insights—

Well. You know where to find us, and we don't know where to find you, or we'd have already reached out.

Hope this helps,

• Duncan

Thank you for doing this work. I think that a graphical representation of the scope of the challenge is an excellent idea, and merits continuous effort in the name of making communication and retention easier.

That being said, I have questions:

1) What is the source of that text document? The citations consist almost exclusively of works concerning nanomachines. None of the citations concern biases, and do not reference people like Bostrom or Kahneman despite clearly being familiar with their work (at least second hand).

2) Am I correct to infer that the divisions along the X and Y axis are your own? Could you comment on what motivates them?

Also, I have suggestions:

Without having read the text document first the numbers confuse, and they are distracting to navigating the image. What do you think of: A, removing the numbers entirely; B, renumbering the text file and the image so the image provides the organization?

What do you think of a way to distinguish between biases that operate individually versus on a group? In example, #51 at (Underestimation, Heuristics) reads "An overly simplistic explanation is the most prominent.", which for an individual could be considered a special case of the Availability Heuristic. Argument against similar problems is found in arguing from fictional evidence, or alternately a form of information hazard. If the prominence of the explanation is the problem, that is a group failing rather than an individual failing.

I also think this warrants a pass for spelling and grammar, but that is merely a question of housekeeping. Would I be right to guess that English is a second language?

Good work!

What do you infer from being able to do this?

(Surely not that qualia are nonphysical, which is the moral Chalmers draws from thinking about p-zombies; colour-blindness involves identifiable physical differences.)

Do they lack qualia? How accurate are these simulations meant to be?

What do people in Chalmer's vein of belief think of the simulation argument?

If a person is plugged into an otherwise simulated reality, do all the simulations count as p-zombies, since they match all the input-output and lack-of-qualia criteria?

I know people who claim that they don't have qualia. I doubt that it is true, but based on their words they should be considered zombies. ))

I would like to suggest zombies of second kind. This is a person with inverted spectrum. It even could be my copy, which speaks all the same philosophical nonsense as me, but any time I see green, he sees red, but names it green. Is he possible? I could imagine such atom-exact copy of me, but with inverted spectrum. And if such second type zombies are possible, it is argument for epiphenomenalism. Now I will explain why.

Phenomenological judgments (PJ) about own consciousness, that is the ability to say something about your own consciousness, will be the same in me and my zombie of the second type.

But there are two types of PJ: quantitative (like "I have consciousness") and qualitative which describes exactly what type of qualia I experience now.

The qualitative type of PJ is impossible. I can't transfer my knowing about "green" in the words.

It means that the fact of existence of phenomenological judgments doesn't help in case of second type zombies.

So, after some upgrade, zombie argument still works as an argument for epiphenomenalism.

I would also recommend the following article with introduce "PJ" term and many problems about it (but I do not agree with it completely) "Experimental Methods for Unraveling the Mind-body Problem: The Phenomenal Judgment Approach" Victor Argonov http://philpapers.org/rec/ARGMAA-2

This looks like an information problem.

It is useful to remember that the market is an abstraction of aggregated transactions. The basic supply and demand graphs they teach us in early econ rely on two assumptions: rational agents, and perfect information.

I expect the imperfect information problem dominates in cases of new products, because producers have a hard time estimating return, and customers don't even know it exists. VCs are largely about developing a marginal information advantage in this space. Interestingly, all of the VCs I have personally interacted with (sample size: 5) say they pick teams over ideas.

When the people at Thinx were asked why the dominant companies hadn't done it already, what was their answer? If they couldn't answer, that would indicate to the VC the team didn't gather enough information to justify their claims (and thus were unprepared). I would expect the answer is some combination of competing with their own products, and demand is not big enough to be profitable with their scaled manufacturing methods.

On the subject of Tums: what is the socially optimal point for sugar-free Tums? How do we know the socially optimal outcome isn't regular Tums and mouthwash?

It is worth keeping in mind that how to defeat X is not well-defined. The usual method for circumventing the planning fallacy is to use whatever the final cost was last time. What about cases where there isn't a body of evidence for the costs? Rationality is just such a case; while we have many well-defined biases, we have few methods for overcoming them.

As a consequence, I determine whether to workaround or defeat X primarily based on how frequently I expect it to come up. The cost of X I find less relevant for two reasons: one, I have a preference against being mugged by Pascal's Wager into spending all my effort on low-likelihood events; two, high cost cases often have a well developed System 2 methodology to resolve them.

A benefit is that frequent cases benefit more easily from spaced repetition and habit forming. In this way, I hope to develop a body of past cases to refer to when trying to plan for how long defeating future X will take.

Examples of frequent cases: exercise, amazon purchases, reading articles. Examples of rare cases: job benefits, housing costs, vehicle purchases.

Military bonding is an interesting comparison. Training in a professional military relies on shared suffering to build a strong bond between the members of a unit. If we model combat as an environment so extreme that vulnerability is inescapable, the function of vulnerability as a bonding trait makes sense.

It also occurs to me that we almost universally have more control over how we signal emotions than how we feel them. The norm would therefore be that we feel more emotions than we show; by being vulnerable and signaling our emotions, other people can empathize instinctively and may feel greater security as a result.