The Poisson distribution is the distribution that models rare independent events.
Are number of fighter pilot victories, clearly, a priori, going to be independent events? That a pilot shooting down one plane is entirely independent of whether they go on to shoot down another plane? (Think about the other two distributions I mentioned and why they might be better matches...)
Distributions are model assumptions, to be checked like any other. In fact, often they are the most important and questionable assumption made in a model, which determines the conclusion; a LW example of this is Karnofsky's statistical argument against funding existential risk, which driven entirely by the chosen distribution. As the quote goes: 'they strain at the gnat of the prior who swallow the camel of the likelihood function'.
I personally find choice of distribution to be dangerous, which is why (when not too much more work) in my own analyses I try to use nonparametric methods: Mann-Whitney u-tests rather than t-tests, bootstraps, and at least look at graphs of histograms or residuals while I'm doing my main analysis. Distributions are not always as one expects. To give an example involving the Poisson: I was doing a little Hacker News voting experiment. One might think that a Poisson would be a perfect fit for distribution of scores - lots of voters, each one only votes on a few links out of the thousands submitted each day, they're different voters, and votes are positive count data. One would be wrong, since while a Poisson fits better than, say, a normal, it's grossly wrong about outliers; what actually fits much better is a mixture distribution of at least 3 sub-distributions of Poissons and possibly normals or others. (My best guess is that this mixture distribution is caused by HN's segmented site design leading to odd dynamics in voting: the first distribution corresponds to low-scoring submissions which spend all their time on /newest, and the rest to various subpopulations of submissions which make it to the main page - although I'm not sure why there are more than 1 of those).
So no, I hope it is because of, rather than despite, my involvement with stats that I object to Rolf's casual assumption of a particular distribution to create a fully general counterargument to explain away data he has not seen but dislikes.
Are number of fighter pilot victories, clearly, a priori, going to be independent events?
Rolf addressed that point:
That is even before taking into account pilot skill, which for all we know has a very wide range.
In particular notice that any deviations from Poisson are going to be in the direction that makes Rolf's argument even stronger.
It is the beginning of a new year, and time for the beginning of a new rationality quotes thread.
The rules are: