= 27d567bc2d48d9f40e9ccc20ea370b15)
Eliezer_Yudkowsky comments on Frequentist Statistics are Frequently Subjective - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (81)
There can also be difficulties with plain data size. Your average journal is not going to publish any tables of a few hundred GB-worth of data; and while you can certainly link to a place to download the files, how long are they going to sit there? It would be rather embarrassing if someone read your paper in ten years and your data server was no longer there.
The notion of changing your mind about the experimental procedure, and thereby changing the significance of the result, is a bit of a straw man. You establish the experimental procedure, then you run the test; your state of mind at the time you flipped the coin is a perfectly ordinary fact about the world, which can influence your priors in a nicely Bayesian way. Of course it's possible to cheat and lie about what your state of mind actually was, but that's not a problem of frequentist mathematics.
How do you think it influences the priors?
If you intend to flip the coin six times, then your null-hypothesis prior is "I will get 0 heads with probability 0.5^6, 1 head with probability 6*0.5^6, and so on". If you intend to flip until you get a tail, the prior is "Probability 0.5 of one flip, 0.25 of two flips", and so on.
That's the likelihood under p = 0.5, not the prior. We want to infer something about p, so the prior is a distribution on p, not on the data.
Sorry, I was confused. Let me try to rephrase. Given some prior, your state of mind before the experiment affects your prediction of the outcome probabilities, and therefore informs your evaluation of the evidence. I should perhaps have said "affects the posterior" rather than "the prior".
The exact example you've given (binomial versus negative binomial sampling distribution) is actually a counterexample to the above assertion. Those two distributions have the same likelihood function, so the evaluation of the evidence is the same under both scenarios. It's true that the prior predictive distributions are different, but that doesn't affect the posterior distribution of the parameter.
Really? I find that counterintuitive; could you show me the calculation?
Suppose that there are two sampling distributions that satisfy (sorry about the lousy math notation) the proportionality relationship,
Pr1(data | parameter) = k * Pr2(data | parameter)
where k may depend on the data but not on the parameter. Then the same proportionality relationship holds for the prior predictive distributions,
Pr1(data) = Integral { Pr1(data | parameter) * Pr(parameter) d(parameter) }
Pr1(data) = Integral { k * Pr2(data | parameter) * Pr(parameter) d(parameter) }
Pr1(data) = k * Integral { Pr2(data | parameter) * Pr(parameter) d(parameter) }
Pr1(data) = k * Pr2(data)
Now write out Bayes' theorem:
Pr(parameter | data) = Pr(parameter) * Pr1(data | parameter) / Pr1(data)
Pr(parameter | data) = Pr(parameter) * k * Pr2(data | parameter) / (k * Pr2(data) )
Pr(parameter | data) = Pr(parameter) * Pr2(data | parameter) / Pr2(data))
So it doesn't matter whether the data were sampled according to Pr1 or Pr2. You can check that the binomial and negative binomial distributions satisfy the proportionality condition by looking them up in Wikipedia.
Your argument is convincing; I sit corrected.