LESSWRONG
LW

Comment Permalink
Idan Arye10

I wonder, though - maybe there are some rational skills that do benefit from repetitive practice? Overcoming bias comes to mind - even after you recognize the bias, sometimes it still takes mental energy to resist its temptation. Maybe katas could help there?

Reply
See in context
by ChristianKl
1 min read
A
735

9

It's my impression that various rationalist groups formed what they called dojos with the intend of having a structure that's similar to a martial arts dojo.

One of the key aspects of that institution is the practice of katas. Katas are a series of moves that can be practiced together to learn a particular art.

As far as I can see we didn't find anything that fits into that spot.

Why didn't we? Is it because rationality inherently can't be broken down into that form? Is it because we approached the problem wrongly?

Meetups & Local Communities (topic)2Techniques2Rationality6
Frontpage

9

Mentioned in
Why didn't we find katas for rationality?
New Answer
New Comment


7 Answers sorted by
top scoring

Matt Goldenberg

200

Possibly because Katas aren't a very good framework for practice?   Most functional martial arts that work in e.g. the UFC will  suggest at the very least practicing with a partner, and gradually working up to live resistance.  Martials arts that emphasize katas over live resistance tend to not be great at self-defense.

My guess is there's something similar going on with rationality. Any sort of kata that doesn't have a heavy component of "gradually working up to interacting with the real world" probably won't be very effective.

[-]jimmy40

While it's true that live resistance is an essential component, it's important to note that effective martial arts do katas as well under different names. In wrestling, it's "drilling". In boxing, it's hitting the bag or pads.

Reply
4Yoav Ravid
The main difference between Katas and Drilling (at least most of the time) is that Katas are usually long chains of moves, and drills are a repetitive practice of a single, short move. Another difference is drills are more often practiced with another person rather than on the air (even hitting a bag is still better than hitting air). Krav Maga style defenses are in the middle, they don't have you do lots of moves out chained together of context in the air, but are also not just one move, they're usually ~5 moves intended for a specific situation practiced on another person with slight resistance. 
4jimmy
The gap isn't nearly so clear cut. In wrestling, we'd do solo drills of shots/sprawls/sit outs/etc every day, and sometimes combine them into chains like sit out->granby->stand up-> shot. The karate guys next to us did their partner katas all the time.  The difference is that when karate dude hits air, it looks like a joke. For all you can tell watching it, not only has this dude never been in a real fight, neither have any of his instructors or his instructors instructors. When Mike Tyson hits air, it's terrifying, and you can tell just from watching it that this guy is very in touch with what is required for knocking out skilled opponents. It's more about whether the connection with the ultimate test is still there and informing how the katas are done than whether you're doing it alone or for multiple moves in a row or anything.
[-]Pattern20

So katas are more 'exercise' than mastery?

Reply
2Matt Goldenberg
Well, they're mastery of a different thing. They're great for entertainment and cool looking demonstrations.

habryka

130

I actually feel like a bunch of forecasting stuff got surprisingly close to being katas for rationality. Like "predicting one Metaculus question a day" seems like a good kata to me, though I might be misunderstanding the concept.

[-]ChristianKl40

A kata contains a bunch of individual moves that are usually done in the same order every time and practiced together.

The idea is by practicing them that way the individual moves will go into muscle memory and executed in a fast way.

Predicting a metaculus question would be a general task that can be accomplished in a lot of different ways.

Reply
[-]AnthonyC130

This is very true, but I think it misses a key point in what makes katas useful for actually learning a martial art in the first place. As noted in Matt Goldenberg's answer, partner work is much more important for actually learning to use a martial art. Just practicing a kata by rote may look pretty, but it won't tell you anything about how to use it. My own best teachers would teach moves and combinations and katas by rote at first, then very quickly move on to exercises that require creative application. Things like: 

  • Mushin practice - get attacked and respond quickly, and make it work
  • Randori practice - get attacked repeatedly by multiple opponents, and make it work
  • Sparring and grappling practice

All of these can be modified with constraints to make them easier or harder. Easier might be "Every attack against you will be this kind of punch." Harder can be something like "Choose one part of one kata. Come up with a way to use it effectively against whatever your opponent decides to throw at you." Or "Figure out X different ways to use that same sequence of moves, with only minor variations, in different situations, then execute it and see how well it works." Hopefully you've be... (read more)

Reply
[-]Ray Doraisamy10

Second this. This is where the katas are. 

Also seconding Matt Goldenberg's point on katas. A kata is akin to learning by rote memorization for an academic test instead of learning to approximate causal models on the fly for use in practical projects. 

Reply

rsaarelm

110

I think it's an issue of "inside the box" vs "open-ended" fields, that we don't have really good vocabulary to talk about. 'Katas' work great for sports that are very much inside the box. You can innovate new strategies, but the rules of the sport set up an unchanging microworld that you must stay inside of. Coincidentally, these are also areas where even current-day AIs often dominate. Established scientific disciplines with research programs are sort of half and half. You can train people in them, but they can also benefit from serious paradigm shifts and there aren't any a priori hard and fast rules about things that absolutely can't be done, like the rules of chess for the chess-playing domain.

Then there's proto-science when things haven't coalesced into a discipline yet, philosophy when it hasn't been professionalized to death, Kegan's stage 5. This is raw pattern matching, flashes of insight, original seeing, very open ended exploration of an unknown landscape. I don't think anyone has had much of an idea for how to systematically train people for this. This place is also where a lot of the actually efficient rationality practice lives.

romeostevensit

80

I did wind up with some personal katas.

Calibration using search: anytime I am searching for something with a quantitative answer I have the chance to do a fermi estimate/reference class forecasting and getting feedback on how I did.

Selection effects/Straussian readings: trying to figure out what incentives drove a particular piece of information to be in front of me in this moment.

Stack trace: finding the provenance of internal maps and noticing that they are often predicated on extremely sparse data which is then overgeneralized.

Schematic thinking: An extension of narrative fallacy bias. Noticing when alternatives would be equally valid when replacing parts of arguments. The implied degrees of freedom make the proposed explanation weaker than it might otherwise seem.

[-]ChristianKl20

Calibration using search: anytime I am searching for something with a quantitative answer I have the chance to do a fermi estimate/reference class forecasting and getting feedback on how I did.

Do you have an explicit process for this?

Reply
2romeostevensit
Same as posts on fermi estimates. I just make a guess to whatever level of effort seems appropriate for the query (often pretty casual, but will take a bit more time if it's about something I feel is important or especially uncertain about). Then when I am getting the piece of info I can reflect on reasons I might have been off. This often helps structure my inquiry into the piece of info also as I map out model differences that are why it surprised me.
2ChristianKl
Do you write the guess down somewhere or just keep it in your head?
2romeostevensit
Written down if I need to multiply a few values to get a ballpark. In head if it's just a direct guess.

steven0461

60

Why aren't there katas for diet? Because diet is about not caving to strong temptations inherent in human nature, and it's hard to practice not doing something. Maybe rationality is the same, but instead of eating bad foods, the temptation is allowing non-truth-tracking factors to influence your beliefs.

[-]steven0461100

People like having superpowers and don't like obeying duties, so those who try to spread rationality are pressured to present it as a superpower instead of a duty.

Reply
[-]Viliam20

Because diet is about not caving to strong temptations inherent in human nature, and it's hard to practice not doing something.

I could imagine an exercise consisting of walking amidst hundreds of delicious cakes, where you know that if you start eating them, no one will stop you...

A more cruel exercise would be having dozens of unknown meals (for example, spheres of blended matter, colored by random food colors) that you are supposed to taste, eat the healthy ones, and spit out the unhealthy ones. (To make it simple, the healthy ones are vegetables, dairy,... (read more)

Reply

Jay

30

Math problems are like "katas" for rationality.  The difference is that, once you've solved a problem once with rationality, you can solve it again much more easily from memory without engaging your rational facilities again.  Therefore you don't get the benefit from repeating the same exercises again and again.

[-]Pattern20

0. You assume there's one optimal solution, and that it has already been found, i.e. no room for improvement, and that nothing changes that would affect the solution.


1. You can change the math problem. (Coming up with or finding novel new problems is also useful, but with work, an old problem can be extended. You've heard of the Monty Hall problem, but how does it change if you have more doors, and more time with doors being opened? If another person was playing a similar game, given the option, would you gain in expected value if switched with them?)

2. Yo... (read more)

Reply
1Jay
Varying the problem helps, as does varying your approach to the problem.  Studying math generally involves many years of working progressively complex problems.  But this is different from a "kata", which is a set of moves rigorously repeated in a specific order and invariant manner*. Psychologically speaking, a kata functions to take a set of moves that the student consciously understands and build muscle memories that can execute the moves effectively at the sub-second timescale of a fight. Reasoning uses different cognitive systems, although once a behavior is consciously understood it is often useful to practice until mental subroutines are developed that enable quick, unreflective execution of the behavior. *There will be some variation as the student goes from not being good at the kata to being very good at the kata, but that variation is unwanted.  The better you are at the kata the less variation there is.
[-]Olomana10

I do sudokus.  These are computer-generated, and of consistent difficulty.  so I can't solve them from memory.  Perhaps something similar could be done for math or logic problems, or story problems where cognitive biases work against the solutions.

Reply
1Jay
Perhaps, but it would surprise me if you don't have hundreds of common sudoku patterns in your memory.  Not entire puzzles, but heuristics for solving limited parts of the puzzle.  That's how humans learn.  We do pattern recognition whenever possible and fall back on reason when we're stumped.  "Learning" substantially consists of developing the heuristics that allow you to perform without reason (which is slow and error-prone).
1Olomana
Right, and if doing computer-generated sudokus is a kata for developing the heuristics for doing sudokus, then perhaps solving computer-generated logic problems could be a kata for developing the heuristics for rationality.
4ChristianKl
The problem is that to be good at rationality you need to be good at interacting with the real-world with all it's uncertainty. 
1Jay
I think we need to distinguish between some related things here: * Rote learning is the stuff of katas, multiplication tables, etc.  It's not rationality in itself, but reason works best if you have a lot of reliable premises. * Developing heuristics is the stuff of everyday education.  Most people get years of this stuff, and it's what makes most people as rational as they are. * Crystallized intelligence it the ability to reason by applying heuristics.  Most people aren't very good at it, which is the main limitation on education.  AFAIK, we don't know how to give people more. * Fluid intelligence is the ability to reason creatively without heuristics.  It's the closest to what I mean by "rationality", but also the hardest to train. * Executive function includes some basic cognitive processes that govern people's behavior.  Unfortunately, it is almost entirely (86-92%) genetic.
1Olomana
Interesting.  I think of heuristics as being almost the same as cognitive biases.  If it helps System 1, it's a heuristic.  If it gets in the way of System 2, it's a cognitive bias. Not a disagreement, just an observation that we are using language differently.
1Jay
I basically agree.  A heuristic lets System 1 function without invoking (the much slower) System 2.  We need heuristics to get through the day; we couldn't function if we had to reason out every single behavior we implement.  A bias is a heuristic when it's dysfunctional, resulting in a poorly-chosen System 1 behavior when System 2 could give a significantly better outcome.   One barrier to rationality is that updating one's heuristics is effortful and often kind of annoying, so we always have some outdated heuristics.  The quicker things change, the worse it gets.  Too much trust in one's heuristics risks biased behavior; too little yields indecisiveness.
[-]Idan Arye10

I wonder, though - maybe there are some rational skills that do benefit from repetitive practice? Overcoming bias comes to mind - even after you recognize the bias, sometimes it still takes mental energy to resist its temptation. Maybe katas could help there?

Reply
2Pattern
It's also helpful to practice when you're first learning something.
1Jay
I can think of a few skills that, while not "rationality" in themselves, make it much easier to reason effectively.  Numeracy is one.  The innumerate can't really see the difference between a million, a billion, a trillion, and a godzillion.   It helps to have, in memory, a set of references to compare to.  For example, there are about a third of a billion people in the United States.  Therefore a billion dollars is roughly $3 each, a trillion dollars is roughly $3,000 each, and a million dollars is roughly nothing (.3 cents) each. A working knowledge of history is also helpful, as is a rough understanding of manufacturing.

DirectedEvolution

20

John Wentworth and asjayan’s framing exercises seem to fit the bill, but I’m the only one consistently participating as of late…

[-]ChristianKl20

It's my impression that John regularly provides different exercises and that there's no continues practice of the same task.

Reply
2DirectedEvolution
In my opinion, what we focus on is sort of like SCUBA diving. Some overall training is helpful, so that you can understand what the equipment is for and how to use it. It does decompose into parts. But it's not a high-pressure sport, and skill comes from mainly going through the overall process and dealing with the random challenges that come up in individual circumstances. So you could teach it as "katas," but that would be a relatively inefficient way to learn.
Moderation Log
A
70