Here's an internal dialogue I just had.
Q: How do we test rationality skills?
A: We haven't come up with a comprehensive test yet.
Q: Maybe we can test some part of rationality?
A: Sure. For example, you could test resistance to akrasia by making two contestants do some simple chores every day. The one who fails first, loses.
Q: That seems like a pointless competition. If I'm feeling competitive, why would I ever skip the chores and lose?
A: Whoa, wait. If competitiveness can cure akrasia, that's pretty cool!
Now we just need to figure out how to make people more competitive in the areas they care about...
For extremely rare events, even anecdotal evidence can be pretty good. If you were told something happened to 1 in a billion people, and you met 3 such people, you'd have pretty good evidence that something odd is going on (the rate is a lie, you're mistaken about meeting them, or something unusual is concentrating them in your presence). Rhabdo isn't nearly that rare, but it's pretty rare. That rareness, and a very reliable base rate for rhabdo in the overall population, means that the anecdotes actually get us a lot further than you might expect. I will demonstrate with some Fermi estimates and a little bit of statistical reasoning:
Wikipedia says:
262m people in the USA in 1995, so the base rate is: 26000 / 262764948
Alright, so that's the base rate for the general population. Now we need a rate for CrossFitters, number of CF rhabdo cases divided by number of people doing CF:
How many anecdotes are we dealing with? I took Nancy's Medium link, followed each link in it (ignoring any other links in this LW thread), and tallied how many specific, concrete, anecdotal cases of rhabdo were reported in each; sorted by date:
http://www.endofthreefitness.com/rhabdomyolysis-know-thyself/ September 2013 : 1
These all sound reasonably exclusive (the 2011 CF paper is specifically about a different time period than the cumulative-to-2005 CF estimate of 6, the comments all seem to be by different people, etc), so we can sum to get 5+2+1+1+4+1+1=15 anecdotal cases of CF. You can argue that some of this is double-counting that I just didn't notice, but if I looked at a few more threads I could probably add some more aneccdotes, so let's just call it even.
How many CFers?
Crossfit was founded in 2000; there are now 6100 affiliated gyms. The 2013 CrossFit Games Open apparently had 138k registrations; this was a sort of virtual online competition, it seems, so it seems like a reasonable proxy for total Crossfit participants. Let's say half didn't bother to register, so in 2013, there were really more like 300k CrossFit people. That makes 50 CrossFitters per affiliate, which given the high cost of CrossFit and intensive time-demanding training sounds pretty reasonable. How many CrossFit people have there ever been? Making the completely arbitrary assumption that the average CFer lasts one year and assuming linear growth from 1 person in 2000 to 300k in 2013, then there's been 150k * 13 = 1950000 CFers.
As it happens, (15 / 1950000) < (26000 / 262764948). To get the CF rate just barely above the base rate, I'd have to increase the number of CF rhabdo cases to 193, since (193 / 1950000) > (26000 / 262764948).
So if you believe the null hypothesis that CFers suffer rhabdo at the exact same rate as the general population, then according to this set of numbers, I apparently managed to get 8% (15/193) of the way to the entire population of CF rhabdo cases just by reading 7 links. As much as I admire my own reading and link-clicking skills, I don't find that very plausible.
Now, with this set of numbers, how many cases of CF rhabdo would we need before we had something statistically-significant? We'd need another 206 cases of rhabdo to hit p<0.05:
So our little handful of anecdotes gets us 7% of the way to the claimed conclusion. For the much-maligned anecdote, this seems pretty impressive, IMO.
Okay.
CrossFit is not that unique. It is just a type of working out hard.
I'm quite confident injuries result more frequently among those who work out hard.
I'll concede rhabdo is more common among CrossFitters than the general population. The point I should have focused (exclusively) on is that that is exactly what we should expect.
We see lots of concussions in football; lots of tennis elbow among tennis players; shoulder issues with powerlifters; knee problems in runners, etc.
The article seems to me to be fear mongering. It is the nature of highly competiti... (read more)