Epistemic Viciousness
Previously in series: A Sense That More Is Possible
Someone deserves a large hattip for this, but I'm having trouble remembering who; my records don't seem to show any email or OB comment which told me of this 12-page essay, "Epistemic Viciousness in the Martial Arts" by Gillian Russell. Maybe Anna Solomon?
We all lined up in our ties and sensible shoes (this was England) and copied him—left, right, left, right—and afterwards he told us that if we practised in the air with sufficient devotion for three years, then we would be able to use our punches to kill a bull with one blow.
I worshipped Mr Howard (though I would sooner have died than told him that) and so, as a skinny, eleven-year-old girl, I came to believe that if I practised, I would be able to kill a bull with one blow by the time I was fourteen.
This essay is about epistemic viciousness in the martial arts, and this story illustrates just that. Though the word ‘viciousness’ normally suggests deliberate cruelty and violence, I will be using it here with the more old-fashioned meaning, possessing of vices.
It all generalizes amazingly. To summarize some of the key observations for how epistemic viciousness arises:
- The art, the dojo, and the sensei are seen as sacred. "Having red toe-nails in the dojo is like going to church in a mini-skirt and halter-top... The students of other martial arts are talked about like they are practicing the wrong religion."
- If your teacher takes you aside and teaches you a special move and you practice it for 20 years, you have a large emotional investment in it, and you'll want to discard any incoming evidence against the move.
- Incoming students don't have much choice: a martial art can't be learned from a book, so they have to trust the teacher.
- Deference to famous historical masters. "Runners think that the contemporary staff of Runner's World know more about running than than all the ancient Greeks put together. And it's not just running, or other physical activities, where history is kept in its place; the same is true in any well-developed area of study. It is not considered disrespectful for a physicist to say that Isaac Newton's theories are false..." (Sound familiar?)
- "We martial artists struggle with a kind of poverty—data-poverty—which makes our beliefs hard to test... Unless you're unfortunate enough to be fighting a hand-to-hand war you cannot check to see how much force and exactly which angle a neck-break requires..."
- "If you can't test the effectiveness of a technique, then it is hard to test methods for improving the technique. Should you practice your nukite in the air, or will that just encourage you to overextend? ... Our inability to test our fighting methods restricts our ability to test our training methods."
- "But the real problem isn’t just that we live in data poverty—I think that’s true for some perfectly respectable disciplines, including theoretical physics—the problem is that we live in poverty but continue to act as though we live in luxury, as though we can safely afford to believe whatever we’re told..." (+10!)
One thing that I remembered being in this essay, but, on a second reading, wasn't actually there, was the degeneration of martial arts after the decline of real fights - by which I mean, fights where people were really trying to hurt each other and someone occasionally got killed.
In those days, you had some idea of who the real masters were, and which school could defeat others.
And then things got all civilized. And so things went downhill to the point that we have videos on Youtube of supposed Nth-dan black belts being pounded into the ground by someone with real fighting experience.
I had one case of this bookmarked somewhere (but now I can't find the bookmark) that was really sad; it was a master of a school who was convinced he could use ki techniques. His students would actually fall over when he used ki attacks, a strange and remarkable and frightening case of self-hypnosis or something... and the master goes up against a skeptic and of course gets pounded completely into the floor. Feel free to comment this link if you know where it is.
Truly is it said that "how to not lose" is more broadly applicable information than "how to win". Every single one of these risk factors transfers straight over to any attempt to start a "rationality dojo". I put to you the question: What can be done about it?
Comments (37)
I believe the video you were looking for is here:
http://www.yachigusaryu.com/blog/2007/02/no-touch-knockout-fraud-exposed.html
http://www.youtube.com/watch?v=gEDaCIDvj6I
" Kiai Master vs MMA "
That wasn't it, but it's even scarier than the one I saw, so ++.
The best study I know of that addresses rationality in pro sports is Moneyball, by Michael Lewis. It tells the story of how Billy Beane followed the approaches recommended by people who studied the voluminous statistics on Baseball and pointed to non-standard evaluations of what talents and strategies made a difference in getting to the post-season. It's relevant for two reasons.
1) It talks about the psychology of players and coaches who found reasons to stick with the tried-and-true, even when non-standard approaches had some evidence in their favor.
2) it talks about the process of re-analyzing the statistics to figure out what aspects of the game matter. Part of this is deciding what the goal is, and part is figuring out what helps you reach the goal. In the case of baseball, Beane agreed with the those who argued that getting to the post-season cost-effectively was the goal. That means figuring out how to win more games over a season, which is more straightforward than figuring out how to win individual games. Cost-effectiveness translates to recruiting players whose value is higher than what other teams are willing to pay. Many unconventional styles of play turned out to be valuable, which led to a team that looked bizarre by accepted standards, but who won consistently but unspectacularly.
edited to use proper LW linking
What can be done about it? We can fight.
The Master can argue for Creationism, and try to defeat the pupil's refutation of it. We can argue for or against One-boxing on Newcomb's problem. Or pretend to be the AI arguing that the Gatekeeper should free it. The Master is only Master for as long as s/he is undefeated.
This equates rationality with victory in argument on an arbitrary side of an issue regardless of the truth; which is not at all the skill we want to inculcate.
Maybe instead of a fight, form it as a riddle:
The master gives an argument for creationism. The "homework" is for the student to understand why this argument is invalid.
Every now and then, just to mix things up, the master would give an argument for a statement which actually turns out to be true, to make sure that the student is actually searching for truth, and not just arbitrary counter-arguments to whatever it is the master said.
It depends upon an empirical question - do more rational arguments win? I think most of the folks around here assume they don't. But if they do, then it sounds like a good enough test.
Most of the points carry over to other domains as well (e.g., music, art, ballet, stage acting, spiritual traditions that have "gurus" or "masters").
For example, there are many (e.g.) piano teachers who can trace their lineage back to Beethoven (and they know it off the top of their heads if you ask them), who are similarly overly deferential to historical masters, who see their knowledge and music in general as sacred knowledge. There is also the same extreme conservatism, and different teaching techniques and performance techniques cannot easily be tested.
[Edit: a pretty good test for whether these sorts of problems are characteristic of at least some practitioners of a given domain is whether (or how often) they get angry in the way preachers get angry at blasphemy and utter sentences that begin with "how dare (s)he ...".
Can anybody think of a domain where students spend decades learning, often with the same teacher or very few teachers, where the domain is the center of their life, which has existed for at least a few centuries, and where these problems do not occur with great frequency?]
By far my biggest problem with the way you discusses rationality is the way that you draw on the tropes of Eastern martial arts instruction, and it's because of exactly this sort of thing - those tropes are appropriate for one who wants to be considered a guru, which is the opposite of your stated aims. It's something I have to warn people about if I'm recommending something you've written.
What successful(as winning in the UFC) martial arts have in common: realistic sparring. BJJ(Brazilian Jiu-Jitsu), Boxing and Muay-Thai all have this(of course with certain restrictions) and they have shown to be effective.
Edit: translating to the rationality dojo, we need some method of empirically testing the techniques. One way would be to do some exercises after the theory was learned like asking some question or doing some test and then seeing who gets the answer right. Another would be to ask for real-life experiences like: what irrational believe or habit did you manage to get rid of in the last month/week? Where did you improve something?
A lot of dojos preserve to some degree the social standards of Eastern countries where the sensei's sensei came from. And in Eastern countries, it's much less acceptable to try to question your teacher, or change things, or rock the boat, or show any form of weakness. I taught school in Japan for a while, and the first thing I learned was that naively asking "Any questions?" or "Any opinions on this?" or even "Anyone not understand?" was a waste of time.
Western cultures are a lot better at this, but not ideal. There's still pressure not to be the one person who asks all the questions all the time, and there's pressure not to say anything controversial out of the blue because you lose more status if you're wrong than you gain if you're right. I think part of the problem is that there really are dumb or egotistical people who, if given the chance will protest that they know a much better way to do everything and will waste the time of everyone else, and our society's decided to .make a devil's bargain to keep them under control.
The best solution to this is to found a new culture, live isolated from the rest of the world for a century developing different cultural norms, and then start the rationality dojo there. Of possible second-best solutions:
My Favorite Liar. Tell people that you're going to make X deliberately incorrect statements every training session and they've got to catch them.
Clickers. One of my lecturers uses these devices sort of like remote controls. You can input information into them and it gets sent wirelessly and anonymously to the lecturer's laptop. The theory is that if he says "Raise your hand if you don't understand this" or even "...if you disagree with this", no one will, but if he says "Enter whether or not you understand this into your clicker" he may get three or four "don't understand" responses. Anonymous suggestion boxes are a low-tech form of the same principle.
I always found the concept of Crocker's Rules very interesting. I also remember hearing of a community (wish I could remember which) in which it was absolutely forbidden to give negative feedback under certain circumstances, and the odd social dynamics that created. In a dojo-like setting, there might be situations when either of these two rules could be ritually enacted - for example, a special Crocker Hat, such that anyone wearing that hat was known to be under Crocker's Rules, and a special No Negative Feedback Hat (but with a flashier name, like White Crane Hat of Social Invincibility), which someone could wear when questioning the master or something and be absolutely immune to any criticism.
I am living (and about to leave) an Asian society very much like this. It yields some very odd results indeed: corruption, consumerism, lemming-like religious behavior, and vast - feudal - social gaps.
In defense of martial arts I want to note that in their case the mentioned key observations for epistemic viciousness are not independend and partially unavoidable. Especially the "deference to famous historical masters" point is no more than a logical conclusion of the "we live data poverty" point: Until less than a hundred years ago people would risk their health (even life) in fights to establish the superiority of the brand of martial artists they espoused. Not to mention life circumstances in general, which more often than not included more or less regular violent conflicts, which usually meat hand-to-hand combat (as opposed to guns). These people did certainly not live in data poverty regarding the efficacy of their techiques, hence it is fair to assume they were (at least on average) better fighters than today's martial artists.
I don't know much about American professional sports--even less about pro sports in other countries--for that matter, I don't know much about martial arts. But as far as I do know, pro sports have none of these problems. Athletes do all sorts of outrageous things; coaches, athletes, and strategies are chosen on merit; absurdly detailed statistics are collected. Baseball players admire Babe Ruth but they don't idolize him. The analogy between pro sports and martial arts isn't perfect, but neither is the analogy between martial arts and rationality.
So, what do pro sports have to "keep them honest", that martial arts don't?
Teams of athletes compete in tournaments that directly demonstrate their skills at their sport. In theory, the sport of martial artists is hand-to-hand combat, but martial arts tournaments never allow eye-gouging, biting, and so on. The further the distance between the tournament rules and reality, the less useful the tournament will be. Fortunately, I don't think there's a rationalist equivalent of eye-gouging, so setting up tournament rules should be relatively easy.
An athlete or coach who gives up a pet technique for one that works better will be rewarded with status and money. The culture of pro sports permits athletes to train in entirely different ways from one season to the next, and coaches to change their playbooks whenever they like. Martial arts schools are stagnant by comparison. The money in pro sports comes from fans (directly through sales or indirectly through advertising) and it would take a lot of effort to raise awareness for rationality. But if rationality masters were really so awesome they'd have no trouble getting the money, right?
Pro sports aren't considered a "way of life" the way martial arts are. Athletes move from one team to another and it's not a big deal, but if Bruce Lee had given up Jeet Kune Do during his life, and taken up Shotokan Karate instead, the martial arts world would still be talking about it. It would be like the Pope converting to Wicca. Readers of OB will probably agree with me that rationality should be a way of life; but I hope they'll also agree that no particular school of rationality should be.
Comment edited for suitable URL tags.
Are you insane? Professional team sports are a bastion of epistemic viciousness. A surprising amount of professional athletes and coaches do not have a coherent grasp of why they are able to do what they do, are awful at evaluating themselves and recognize, yet dismiss, what they should do to get better. Case in point: Shaquille O'Neal, with his free throws and rejuvenation once he encountered the Phoenix medical staff.
Or any number of idiotic football coaches who refuse to implement strategies that Madden video games and real life show as valid, winning strategies. On the other hand, there's Don Nelson - who appears to be playing a demented brand of basketball in a bizarro dimension.
Disclosure: I have done Brazilian Jiu Jitsu for eight months and dabble in mixed martial arts. I have watched more than a few hundred hours of videos of all kinds of martial arts, been active in individual and team sports and did taekwondo a long time ago.
My experiences in BJJ and MMA have shown me a population of people unusually aware of the strengths and limitations of almost every martial art out there. There's a strong institutional emphasis (from the instructor) to do techniques shown in class specifically as shown; however, there's also a strong unofficial emphasis on watching YouTube videos, grappling with other people and coming up with stuff on your own time. Both pathways are tested in grappling. The OODA loop works so much better within the BJJ/MMA groups than it does in people outside.
I have no idea why this is, but I suspect it is primarily because of the UFC and other MMA organizations showing the continual development of individual combat (within rules). The personal fighting has also borne this out, but isn't nearly as capable of influencing other people.
By continual testing against others, the chinks are eventually shown and either patched up or styles reconfigured. A variety of styles and strategies have been shown to work - swarming (old Shogun, old Wanderlei), counterfighting (Evans, Rampage), Muay Thai (Anderson Silva), submissions from the top (Maia), submissions from the bottom (Minotauro), wrestling (St. Pierre) etc. [Note: almost all of the previously mentioned are world-class experts in multiple disciplines]
Bruce Lee sorta gave up Kung Fu. Pro sports are a way of life for many, many millions.
Rationality dojo: isn't this place one?
I don't think I'm insane. But then, I would say that, wouldn't I?
You've misread me to suit your preconceptions. I never said that there was no epistemic viciousness in professional team sports. What I said was that the particular problems that Russell describes aren't problems in pro sports. It's possible to learn from the pro sports model without adopting it in every particular.
Of course not all football coaches rationally choose strategies; not all football coaches are competent, period. But unlike the dojos Russell describes, in pro sports that behavior in is understood as biased and unreasonable, not praised as respect for tradition.
I agree that "pro sports" are a way of life for many people--this was phrased poorly in my original post. I should have said that membership in a team isn't a way of life for professional athletes. Fans generally stick with one team or another, but when you move from Chicago to Los Angeles, it's not a big deal if you stop following the Bulls and start following the Lakers. Anyway, the analogy breaks down here--what would a "rationality fan" who didn't actually practice rationality look like?
You say the breadth of martial arts knowledge of your BJJ/MMA community is "unusual." I assume you meant relative to the rest of the martial arts community rather than the general population, which would be trivially obvious. Either way we agree that "continual testing against others" is the common denominator that keeps a dojo or a professional sports team effective.
Yes, I mean relative to the rest of the martial arts community.
"What would a rationality fan who didn't actually practice rationality look like?" Jim Cramer on the Daily Show? (I refer not to the verbal destruction, but Cramer's stated appreciation of Stewart's points without any subsequent change in his behavior.)
Well, the Cornhuskers had a big thing for the Option I offense for a very long time, and recruited talent specifically for it - despite the growing utility of more "modern" offenses. There was a huge hullabaloo about the switch to the West Coast under Callahan. A significant portion of Husker fans still grumble about it, and mostly do so with the "tradition" criticism.
I can't wait until a college or pro team does the A-11 offense: http://highschool.rivals.com/content.asp?CID=825031
Repeat after Tarski:
If I am insane,
I desire to believe that I am insane;
If I'm not insane,
I desire to believe that I'm not insane;
Let me not become attached to beliefs I may not want.
Well, then again, I don't think there's a rationalist equivalent of a tournament just yet, either.
Well... there is the stock market, but that's generally too much of a challenge; any edge you get disappears very quickly, so the best thing to do is "free ride" off of other people's attempts to value stocks and just buy index funds (or the equivalent).
Other domains in which rationality can be tested are "intellectual sports" such as poker, chess, or Magic: The Gathering... it's hard to test "rationality" in a way that doesn't simply test intelligence or learned skills, though.
Actually, the martial arts world has recently benefited from a big dose of reality in the form of mixed-martial-arts tournaments. Throwing together fighters from different styles demonstrated that some were overwhelmingly superior to others and unleashed a rapid evolution of technique that blended together the clearly superior methods. http://en.wikipedia.org/wiki/Mixed_martial_arts#Evolution_of_fighters
This is sort-of true, but with one really, really big caveat that people seem to forget: any form of fighting that is controlled basically screws large portions of many styles.
If you go into an MMA tournament and deliberately break someone's arm, you aren't going to be asked back. Let alone if you break their neck. Furthermore, non-crazy martial artists don't even want to: there's too much respect for that. There are styles that are centered around causing maximum damage as quickly as possible, and they are entirely useless in MMA fights. You're never going to see a hard-style master being competitive in an MMA tournament, because 90% of what they know is irrelevant.
-Robin
rlpowell, you are incorrect. You are spouting an untested theory that is repeated as fact by those with a vested interest in avoiding the harsh light of truth.
In actual fact, there is no problem with breaking someone's arm in an MMA fight (see Mir vs. Sylvia in the UFC, for example). It's also close to impossible to break someone's neck (deliberately), despite what you may see in movies.
The "we're too dangerous to fight" is an easy meme to propagate. But let me just ask you this: let's just say, hypothetically, that your theory ("maximum damage" masters are "useless in MMA fights") was false. How would you ever know? Assuming that someone did not yet have a belief about that proposition, what kind of evidence are you actually aware of, about whether the statement is true or false?
There is a pretty simple way to test this, it's simply somewhat dangerous and arguably unethical. Take a "maximum damage" fighter, and send them into a number of no rules fights where they can justify using maximum force, and then pit them against MMA fighters in sanctioned matches.
I don't know of any style that does this, but I did train for a while in a style that does something similar. In Wun Hop Kuen Do (and possibly other branches of Kajukenbo (http://en.wikipedia.org/wiki/Kajukenbo) being an instructor level practitioner is essentially a research position. You're required to test your skills in realistic situations, because as a teacher, the danger to your students if you instruct them poorly takes precedence over the danger to yourself. My sifu, Jason Goldsmith, would have collaborators attack him in earnest with a sharpened knife (he worked his way up from rubber ones,) or fight against multiple opponents, in order to make sure his skills worked where it really mattered.
Sifu Jason does prepare people for competitions, including MMA, if they request it, but he makes it very clear that that isn't really what the style is meant for, and is not the setting in which it's most effective.
This is an aside, but related: there is an awesome website dedicated to investigating and uncovering fraud and vice within the Martial Arts world: Bullshido. Participants post about questionable doings (such as masters who claim to teach or use ki) and visit those schools to report on the fraud. It also has areas for regular Martial Arts chat, but this sub-forum is the one that primarily focuses on investigations.
This, then, is an incomplete outline to an answer to your question: due diligence and active pursuit of those who fraudulently represent "arts of rationality." If there were a series of Dojos of the Bayesian Sect, those dojos would be responsible for exposing the Thousand Schools of False Ways. It would ever be an ongoing battle. Just as the members of Bullshido are constantly encountering and exposing "McDojos" that claim to teach practical self-defense techniques that would, in reality, probably just get you killed or kids' grappling schools taught by instructors with sex assault convictions.
I was going to link to this but you beat me to it. One of the things Bullshido tends to believe and uphold is this concept of "aliveness". It can be thought of as a sort of loose guide to "empiricism/rationality as it applies to physical activities".
I hate his style but Thornton describes aliveness and came up with the term(although the concept has been around pretty much forever I suppose). http://aliveness101.blogspot.com/2005/07/why-aliveness.html
I do BJJ and, as I see people here already know, we tend to do a pretty good job keeping our eyes on the ball. Things are handed down from on high, but it is understood by everyone that each person may need to deviate in various ways to make the move work for them and the true test is "can you make it work in rolling?" Experimentation is encouraged, although a common base is pushed very hard.
Generally people try to perform a move right after having practiced it on other people in the group that are aware and expecting the attempts. This in and of itself helps ensure high quality. The theory is that if you can make it work on someone expecting it it will be easier to catch the unaware.
People tend to be quite silly/irrational about fighting - I think, as I see others here do, that it's because actual feedback and first hand experience is in short supply. "Pure reasoning" leads many into absurdities without a strong empirical base.
Doesn't seem like an "aside" to me; investigations that look for that sort of simple fraud was something I hadn't yet thought of.
Well then, here is a proper aside. (Found in this thread.)
I'll suggest it first so nobody else has to lose karma over it: don't start a rationality dojo.
Is this what you're looking for?
Shut up and do the moderately difficult.
I'm not sure if that's the link, but it seems similar if it's not the one.
The way I test the rationality of the people around me is by lying to them, generally about irrelevant things, and seeing if they can unravel the lies.
In virtually all cases, what happens is that they simply learn not to believe anything I say, at which point I start telling the truth in a way that makes it seem like a lie. People dial their credulity up and down, and eventually just give up.
From the general attitude of the people here, I doubt most of them have gotten beyond the "calibration" mindset either, thinking of rationality like tuning a TV, just trying to correct for the biases. It isn't that simple.
If you want to test your rationality, you have to try and determine the truth about something difficult, and you have to be able to verify the results. This can be done by:
having someone around who knows the answer to begin with, and is putting the problem to you as a test
working on a problem where the correct answer can be verified because it allows you to make predictions of some kind
working on a problem so tricky that even coming up with an answer which is internally consistent is very difficult, i.e. the miracle of the sun
And when they call you on your bull, you say "I was only trying to make you think"? I think I met you at a party once.
I don't believe you.
Me neither. Are the people around you really paying so much attention to you that they would go such effort? Ones who aren't related to you?
"The way I test the rationality of the people around me is by lying to them, generally about irrelevant things, and seeing if they can unravel the lies."
Unless you are carefully documenting your procedures (which I doubt), this is exactly the kind of informal "testing" that gets superstitious or mystically inclined individuals into epistemic hot water. Playing pranks on your friends is one thing, striving for increased human rationality something else.
The test seems about as useful as a thermometer that starts at 100 degrees. It doesn't tell you a lot about your environment except in extreme situations. Not very useful, at least as a first test.