beoShaffer comments on Rationality Quotes January 2015 - Less Wrong

4 Post author: Gondolinian 01 January 2015 02:23AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (148)

You are viewing a single comment's thread. Show more comments above.

Comment author: RolfAndreassen 03 January 2015 10:11:52PM 11 points [-]

In World War Two, it is a fact that only 15-20 percent of the soldiers fired at the enemy.

One of the originals is Men Against Fire by World War I Officer S. L. A Marshall.

You find this claim all over the place; the problem with it is that comrade "S.L.A.M" is not "one of the originals", he is the sole and only source for the claim - and he made it up. A cursory Wiki search shows:

[So-and-so demonstrated] that Marshall had not actually conducted the research upon which he based his ratio-of-fire theory. "The 'systematic collection of data' appears to have been an invention."

My emphasis.

The best evidence we have is that killing is in fact difficult for most people, most of the time, even in war.

Ok. So on the one hand we've got a single book, later shown to have been an invention, but taken up by a huge number of people so it looks like a consensus, in the best Dark-Arts, "you have to be smart to know this", counterintuitive-Deep-Wisdom style. And on the other hand we have a huge amount of dead people, mysteriously killed by bullets that, somehow, got fired in spite of the noted reluctance of men to do so. I propose that your accolade of "best evidence" is a bit misplaced.

This is an excellent example of the need to apply some skepticism to a counter-intuitive but neat-seeming claim, whose possession will put you inside the tribe of people who Know Neat And Counterintuitive Stuff. Sometimes the simple answer really is the right one; this is one of those times.

Comment author: beoShaffer 04 January 2015 12:52:36AM 0 points [-]

Can you please link what you're quoting from.

Comment author: RolfAndreassen 04 January 2015 01:22:29AM 5 points [-]
Comment author: beoShaffer 04 January 2015 01:59:06AM *  0 points [-]

Thanks. -ETA I followed both the link and the links to several of Wikipedia's sources, but no further. The stuff I saw all seems to support Rolf's claims about S. L. A Marshall being unreliable and the primary source for most of the claims of the killing is hard side.

Comment author: gwern 04 January 2015 02:15:29AM 1 point [-]

Isegoria claims Grossman's claims, if not Marshall's, is better supported by things like fighter pilot studies: http://westhunt.wordpress.com/2014/12/28/shoot-to-kill/#comment-64665

Comment author: RolfAndreassen 04 January 2015 02:29:46AM 0 points [-]

Fighter pilot victories in clear-air combat are rare; it follows that they are Poisson-distributed, and that you would expect to have a few extreme outliers and a great mass of apparent "non-killers" even if every pilot was doing his genuine best to kill. That is even before taking into account pilot skill, which for all we know has a very wide range.

Comment author: gwern 04 January 2015 02:51:40AM *  2 points [-]

Fighter pilot victories in clear-air combat are rare; it follows that they are Poisson-distributed, and that you would expect to have a few extreme outliers and a great mass of apparent "non-killers"

I don't see how that follows at all. You don't know it was a Poisson distribution (there are lots of distributions natural phenomena follow; the negative binomial and lognormal also pop up a lot in human contexts), and even if you did, you don't know the the relevant rate parameter lambda to know how many pilots should be expected to have 1 success, and since you're making purely a priori arguments here rather than observing that the studies have specific flaws (eg perhaps they included pilots who never saw combat), it's clear you're trying to make a fully general counterargument to explain away any result those studies could have reached without knowing anything about them. ('Oh, only .001% of pilots killed anyone? That darn Poisson!')

Comment author: alienist 05 January 2015 03:12:51AM 3 points [-]

You don't know it was a Poisson distribution (there are lots of distributions natural phenomena follow; the negative binomial and lognormal also pop up a lot in human contexts),

The Poisson distribution is the distribution that models rare independent events. Given how involved you are with prediction and statistics, I'd expect you to know that.

Comment author: gwern 05 January 2015 04:23:57AM *  3 points [-]

The Poisson distribution is the distribution that models rare independent events.

Are number of fighter pilot victories, clearly, a priori, going to be independent events? That a pilot shooting down one plane is entirely independent of whether they go on to shoot down another plane? (Think about the other two distributions I mentioned and why they might be better matches...)

Distributions are model assumptions, to be checked like any other. In fact, often they are the most important and questionable assumption made in a model, which determines the conclusion; a LW example of this is Karnofsky's statistical argument against funding existential risk, which driven entirely by the chosen distribution. As the quote goes: 'they strain at the gnat of the prior who swallow the camel of the likelihood function'.

I personally find choice of distribution to be dangerous, which is why (when not too much more work) in my own analyses I try to use nonparametric methods: Mann-Whitney u-tests rather than t-tests, bootstraps, and at least look at graphs of histograms or residuals while I'm doing my main analysis. Distributions are not always as one expects. To give an example involving the Poisson: I was doing a little Hacker News voting experiment. One might think that a Poisson would be a perfect fit for distribution of scores - lots of voters, each one only votes on a few links out of the thousands submitted each day, they're different voters, and votes are positive count data. One would be wrong, since while a Poisson fits better than, say, a normal, it's grossly wrong about outliers; what actually fits much better is a mixture distribution of at least 3 sub-distributions of Poissons and possibly normals or others. (My best guess is that this mixture distribution is caused by HN's segmented site design leading to odd dynamics in voting: the first distribution corresponds to low-scoring submissions which spend all their time on /newest, and the rest to various subpopulations of submissions which make it to the main page - although I'm not sure why there are more than 1 of those).

So no, I hope it is because of, rather than despite, my involvement with stats that I object to Rolf's casual assumption of a particular distribution to create a fully general counterargument to explain away data he has not seen but dislikes.

Comment author: alienist 05 January 2015 05:26:59AM 4 points [-]

Are number of fighter pilot victories, clearly, a priori, going to be independent events?

Rolf addressed that point:

That is even before taking into account pilot skill, which for all we know has a very wide range.

In particular notice that any deviations from Poisson are going to be in the direction that makes Rolf's argument even stronger.