That was cute. I'm not sure I understand it, though.

:)

You certainly do take care to respond to the comments.

I'll try to decipher the message.

If I am concerned that the group I belong to is becoming cultish, the thing to do is to ask what is a cult and what is not a cult, and see if the definition applies to your group. The second koan reminds us not to use extraneous details like uniforms. More non-cultists than cultists use uniforms. In general P(Category|Feature) != P(Feature|Category).

I THINK that's what you're saying...

I swear, the grammatical screwup in the above post was completely intentional.

Tiiba, do you mean to imply that if your group is cultish, the group I belong to is safe by the tu quoque principle?

;)

Tiiba, do you mean to imply that if your group is cultish, the group I belong to is safe by the tu quoque principle? ;)

In all seriousness, I think the second point is a little more complicated than you described. Specifically, I think Mr. Yudkowsky is trying to point out that Bouzo had plenty of evidence more closely entangled with cult/non-cult status than whether or not they had uniforms. He had experienced Ougi's teachings directly. In the absence of that evidence, of course, the uniforms would be more important evidence.

But Mr. Yudkowski, why shouldn't we be concerned with such apparently superficial signs of cultishness? Shouldn't we -- especially the less experienced rationalists among us -- be on the lookout for objective clues to bias? After all, as has been pointed out already on Overcoming Bias, situations often look different from the inside, in ways that generate inaccuracies. (People tend to underestimate how biased their beliefs are. It's easier to show that someone's wearing uniform clothes than to show that they're wearing irrationally uniform beliefs.)

...okay, maybe this is a cult.

"I THINK that's what you're saying..."

I think it is saying that if you want to know if an idea is true or not, compare it to reality. Clothes are an irrelevancy. "If it can drive nails, who can doubt it's worth?"

Well, I must confess I was initially arguing in gamelike fashion -- just to see what your next move might be -- but I fell into my own trap door, and I'm really unsure that 1) this isn't a cult, and that 2) that would be a bad thing.

That is, who cares if we have mutually reinforcing behaviors etc.? What matters is whether these are good mutually reinforcing behaviors, and that we evaluate their goodness from a non-tautological (i.e. external) perspective. (That is, I'd distinguish between circles and vicious circles.)

I don't know whether or not that's part of your point. I suppose I shouldn't care. But it's certainly what the idealized Eliezer Yudkowsky in my head thinks. I certainly hope you're as smart as he is. ;)

Ian C.,

If I had a hammer that seemed to me to work really well, but no one was willing to pay me the going rate for hammers of that quality, would it really be ridiculous for me to seriously question the accuracy of my perception of the hammer?

"If I had a hammer that seemed to me to work really well, but no one was willing to pay me the going rate for hammers of that quality, would it really be ridiculous for me to seriously question the accuracy of my perception of the hammer?"

Yes, in the case of a hammer I think it would be [ridiculous to doubt yourself]. In the case of something more complicated, like Wine, you might start to question whether there is some subtle difference between your wine and theirs that you're not detecting, but there is no such thing as a hammer connoisseur.

"If I had a hammer that seemed to me to work really well, but no one was willing to pay me the going rate for hammers of that quality"

.. then by definition you have mis-estimated the going rate.

I think I'll side with the novices against Ougi here. The novices deserve a clearer answer than "think of ways to resolve your doubts" and "all will be clear when you try to use this stuff." Cults usually tell people many things that are actually useful, and confronting leaders seems a reasonable way to resolve doubts. As I said before, the word "cult" is a bit too easy a word to throw around - I'd prefer a clearer description of what it means exactly and how to recognize one.

Robin, transmission of expertise in non-rational domains has to rely on authority rather than argument, so is more susceptible to slide into abuse of authority than transmission in rational domains. The original post here is strange in that it supposes such a type of transmission in the field of rational teaching. The definition of cult in the field of master / disciple relationships has to start with an examination of whether authority is being abused by, for example, being exercised in areas unrelated to the teaching. Don't take sweets from philosophers.

I only got into the rationalist game because I'm fond of hats. Oh, and Robin's advice on gift-giving.

Well, this is a bit better than normal.

If you can immediately recognize the candlelight as fire, the meal was cooked a long time ago.

Ian C., do you know for sure, before evaluating this kind of evidence, whether the problem is simple like you seem to think a hammer is, or complicated like you seem to think wine is? I had thought that wine is simpler as its functionality is a matter of taste, while hammers' functionality is something that exists in some sense outside my first impression of it. (It may be hammering in the nails all crooked-like or something, and perhaps I don't notice bur everyone else does.) What mistake am I making?

Botogol, why not say that I have misestimated either the going rate, or what quality a hammer it is. (If I think it is steel, but it is cast-iron, that doesn't mean I have misestimated the going rate for steel hammers; it means I don't have one.)

Caledonian, I have no idea what that means, but I'm sure it's very profound. I would greatly appreciate it if you or anyone else would explain it.

I had a friend in college who was a philosophy major; he'd been raised a fundamentalist Christian, and turned from that into some sort of chaotic evil deist. I used to enjoy arguing philosophy and ethics with him. But the further he got into his study, the more his arguments turned into half-understood quotations of Wittgenstein and Kant, and his debating technique turned into sophistries and trying to name and call out others' logical fallacies, sometimes correctly and sometimes not -- the same techniques he grew up with, only without having to wake up early on Sunday. I don't argue with him anymore...

I would greatly appreciate it if you or anyone else would explain it.

You cannot understand the moon by deconstructing the finger.

The student asked, "Where is the true master of the Way?" The master replied, "Where is a true student of the Way?"

Benquo: "What mistake am I making?"

I think you are artificially restricting your knowledge to direct perception of the objects. Don't you also know that there are entire shops that sell only wine, that they have many varieties, at many prices, and they have competitions, experts etc?

It is *this* knowledge that leads you to conclude that wine is probably a complicated business and you may very well have made a mistake.

"Since you are so concerned about the interactions of clothing with probability theory," Ougi said, "it should not surprise you that you must wear a special hat to understand."

But isn't this almost the exact opposite of what the student was saying? Questioning the robes indicates to me that the student felt there was not any interaction between learning probability theory and clothing, and that therefore it served some other purpose, presumably differentiating between an in group and an out group.

Or am I just nuts for trying to argue with you about the internal thoughts of your own fictional characters?

Do not attempt to deduce the thought processes of fictional characters; that is impossible. Seek only to recognize the truth: that what the author says is objective, has implications on its own, and does not necessarily have anything to do with what he meant.

A koan: If you have ice cream, I will give it to you. If you have no ice cream, I will take it away from you. It is an ice cream koan.

Caledonian, did you get your's from Stargate?

A few of you touched on the point I got out of this, but no one explained it very well. In the first koan, Ougi says two things. The clearer one is tangential to rationality, but important for self-doubting cultists. "You are like a swordsman who keeps glancing away to see if anyone might be laughing at him".

The more important point was that the teachings are valuable if they are useful. (This is applicable to the sword fighter because allowing yourself to be distracted is an immediate danger.)

The importance of the parable about hammers doesn't relate to prices, but to usefulness. "Use the hammer to drive nails" in a discussion about rationality is metaphoric for using the techniques to make better decisions. If Ougi's teachings help you make better decisions in your life, then they are valuable. If they merely bind you more tightly to Ougi, then you are a cultist.

Bouzo didn't learn anything that helped him make decisions, he was merely cowed into following Ougi more closely. Ni no Tachi learned to "concentrate on a real-world question", so "the worth ... of his understanding [became] apparent."

Ni no Tachi figured out how to use the hammer, but Bouzo only sold them without understanding their value.

Chris, for a while I was a member of what most people would consider a "cult", and from my experience "cults" usually teach people useful things. So it is a mistake to assure yourself you are not in a cult because you see that you are learning useful things.

Caledonian, did you get your's from Stargate?

No. I got it from the same sources as Stargate's writers.

Lightning flashes. Sparks shower. In one blink of your eyes you have missed seeing.

If clothes are unrelated to probability theory, why do the students have to wear robes?

Why not?

Why not just regular everyday clothes?

Robin, you're right, most people do think economics is a cult, even though there may be a small proportion of usefulness in the teachings... characteristics are, cult members cut off from contact with non-cult members (in this case by the ignorance of the non-cult members, of course), devotion to the cult leader (Keynes ! Friedman ! the Gourd! the Sandal!), proclamations of infallibility (the market is infallible), progressive alienation (this is a science, I can believe six impossible things before breakfast), and ending in total learned helplessness (for instance, when a team of six beauticians, or whoever it happens to be this week, outperform the nation's best fund managers yet again...). Only teasing, but I was just reading some very old threads and came across one where you professed surprised at relative levels of acceptance of announcements in economics and physics, and am still suffering from vertigo. Happy Christmas !

"Why not?"

Because it confuses them into thinking that maybe the clothes have something to do with the meat of the instruction, and it takes longer for them to learn what they're being taught. It obscures the the real teaching, hides it among irrelevant rituals.

Ni no Tachi figured out how to use the hammer, but Bouzo only sold them without understanding their value.

"A bird in the hand is worth what you can get for it." --Ambrose Bierce

Fiction is fiction, but it seems to me that that if student objects to wearing silly clothes and his master responds by ordering him to wear yet sillier clothes, it's a lot more plausible that the student will conclude his master is a quack and drop out than that he'll decide to extend his master's teaching by taking silly clothes to a whole new level.

Maybe the whole point of this exercise is to remind us that one can't come to reliable conclusions from fictional evidence? If so, well, maybe I haven't learned anything... but at least I've learned I haven't learned anything.

Well, it's clear that a lot of people found the koans confusing. Silly me, I realized I forgot to include Mumon's commentaries!

Mumon's commentary on the first koan:

Because Ni no Tachi hated rich men, Ougi gave up his wealth of silence. When Ni no Tachi became a rich man, He loaned money to his students.

Mumon's commentary on the second koan:

A flower is neither bread nor water, Why not replace it with a weed? Bouzo changed the shape of Ougi's garden, But did he really understand?

I still like Tom's the best.

Davis, thanks, that is good.

I don't think Hanson is saying economics is a cult, he is referring to either this or this.

I think Eliezer outed himself as an anime fan in this post. It's ok, I'm a pretty big one myself.

Thanks TGGP. That's a better suggestion than I came up with. Since Robin seemed to be responding to me (rather than to "Chris"), I guessed he was referring to the "cultish" experience he shared with me. But your suggestions are more likely.

Robin: "It is a mistake to assure yourself you are not in a cult because you see that you are learning useful things." But isn't the opposite conclusion safe? If you can't figure out how to use the hammer, but you nonetheless convince younger students of its importance, aren't you likely to be in a cult? Or more likely, if you can see that many of the other teachers are teaching material they don't understand?

If you can't figure out how to use the hammer, but you nonetheless convince younger students of its importance, aren't you likely to be in a cult? Or more likely, if you can see that many of the other teachers are teaching material they don't understand?

Sounds like either a cult or a college.

What's wrong with being in a cult, as long as you're reasonably self-aware about it? Cults are fun! ;)

Incidentally, Wikipedia lists itself in its list of alleged cults.

Did I mention I hate Zen-style teaching?

Hate hate hate.

Unless you're going to tell me Godel, Escher, Bach consisted of Zen-style teaching, in which case I'll bring it down to two hates. (But it sure is pretty.)

Also, college: hate hate hate.

My least favorite portions of GEB: An EGB were the Zen ones.

Did I mention I hate Zen-style teaching?

I presume you feel the same way about the Socratic method?

Eliezer Yudkowsky wrote of ideas one can't see the value of, and teachers who don't seem to understand their teachings, "Sounds like either a cult or a college."

I dunno, at least for many technical fields and for some other endeavors too (like learning to communicate effectively in writing) one can see that many of the teachers can do some handy hard-to-fake real-world stuff, and that the students emerging through the pipeline tend to be able to do it too. When I was an undergraduate, the EEs in my residence hall traditionally maintained a little hand-made custom-programmed telephone PBX which ran from the two college official phone jacks in the lobby to a motley collection of old salvaged telephones in most of the other rooms. I, at least, was impressed. If you're in an organization where the initiates routinely levitate out their windows to go to lunch, and levitate some more whenever they have trouble finding a convenient chair, is it a mystical cult because levitation or funny hats or even confusing explanations are involved, or might it be unusually successful pragmatic applied philosophy?

Once stretched to cover everything from incompetent posers to arrogant weird competent people (like Isaac Newton at the hypercompetent extreme, or various academics in a less extreme way), a concept like "cult" may not be all that valuable. Perhaps there is value in reminding us that part of the reason the posers can gull people with their behavior is that it's not so uncommon for non-posers to act in some similar way. But there is also value in to reminding people that part of the reason speculative bubbles can happen is that price moves based on fundamentals can look similar enough to gull speculators into mistaking a bubble for one one. That doesn't mean we should think of every big price move as a "bubble" (or as being bubble-ish, or whatever). We might say "every big price move wants to be a bubble," but saying a market situation where the fundamentals don't make sense "sounds like a bubble or an ordinary market" would seem to me to be missing a point.

From the Discworld novel Thief of Time:

In the Second Scroll of Wen the Eternally Surprised a story is written concerning one day when the apprentice Clodpool, in a rebellious mood, approached Wen and spake thusly:

"Master, what is the difference between a humanistic, monastic system of belief in which wisdom is sought by means of an apparently nonsensical system of questions and answers, and a lot of mystic gibberish made up on the spur of the moment?"

Wen considered this for some time, and at last said: "A fish!"

And Clodpool went away, satisfied.

Caledonian, not so much, no, though I've never really noticed being used so much. (If I did maybe it would annoy me too.) Why would you think so?

Because Socratic-style teaching IS Zen teaching, only with training wheels on.

Caledonian, do you mean the sort of stuff found in Plato's writings, or do you mean the question-and-answer style that passes for "Socratic" in law schools etc.?

I can see how the former might be called Zen with training wheels, but not the latter.

Oy. I just glanced through the last couple weeks of posts. Hence the lack of a loud sigh on this one before. So consider this the loud-sigh of the confirmedly anti-koan, the person who thinks that metaphor and other such non-expository modes of speech have aesthetic value only, and that if one cannot speak of an idea in clear language, well, one ought to keep silent about it. (I can see Wittgenstein glaring at me...)

Or: what's the point of rationalist koan* exactly?

* It also irks me like crazy to see people taking the Japanese word "koan" and sticking an s on the end to pluralize it. You don't do that in Japanese.

It must be difficult traipsing around the Highlands in those big clown shoes.

It seems to me that some people take the notion "Never accept anything uncritically!" (or equivalently, "There are no certainties in science!") too far.

The core tenets of logic as set out by Mill, at least, must be accepted uncritically and never doubted, or the whole conversation in which they are doubted disintegrates into fallacy and nonsense, and thus becomes useless (except to a dishonest speaker who might use it to manipulate irrational people). There are other beliefs which are similarly necessary (for instance, mathematics) if the discussion extends to topics where they apply.

Snort. Got here while trying to figure out what, if any answer (the comic provides none) there is to "What kind of ice cream do you put on a Koan?" And here I find this... lol

Seriously, though, like most arguments presented by monk style philosophers, the answer given is flawed. Which is more valuable of the following?

1. A pure gold hammer with a mess of rhinestones in the handle. Estimated material cost - $200.

2. A real hammer, with a gold filigree handle and an idiotic mess of bangles hanging off of it, jewels strung onto them. Estimated material cost - $200.

3. A real hammer, simple black handle, estimated store cost - $10.

Well, if the **point** of the hammer is to build a house, you have a problem, because both 2 and 3 are options. Number 2 is silly as hell, regardless of whether you glued all the junk on it yourself, or you claim to have found it in the lost tomb of Jesus. It may still hammer nails, but no one in their right mind would use it, except out of necessity of there being no other hammers available. Option #1 is completely useless, gold being too soft to hammer much of anything, and again, it doesn't become a more valuable **hammer** if you claim that "it" was found in the lost tomb of the great Illuminati or some similar gibberish. A cult is a cult because it either contains nothing but gold hammers, or it insists that everything **must** be done using hammers that have lots of idiotic junk glued onto them, which don't help make the hammer any more useful. Someone intent on building a house can as easily use the second option as the third, but the moment you start thinking that you "have to" have all the silly BS glued to the hammer for it to work, it becomes a cult. Witness pretty much all religions, which nearly universally insist that its impossible to invent a hammer, without the guidance of high order of hammer makers, as to what sort of bangles and filigree one has to add to it before the magic spirits will make it work right. ;) Some are though tend to be satisfied with just the hammer, but insist the handle must be red, or something, to make it lucky (which might be the approach of the few relatively "rational" religions in the world), of which most are not.

In general, the more rational your starting framework, the least silly the necessary robes/clown suits are, but even the most rational groups, if they don't take a critical look at certain assumptions, can get cultish. The key difference being if they opt to make those choices for distinction, or because they have actually come to *believe* that some detail of their world won't work without the silly hat.

Oh, and anyone know the answer to the original question that got me here? lol