jsteinhardt comments on Original Research on Less Wrong - Less Wrong

21 Post author: lukeprog 29 October 2012 10:50PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (47)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 31 October 2012 02:39:46PM 2 points [-]

the significance of some evidence E is actually the probability that the null hypothesis is true, given E

No frequentist says this. They don't believe in P(H|E). That's the explicit basis of the whole philosophy. People who talk about the probability of a hypothesis given the evidence are Bayesians, full stop.

Statistical significance is, albeit in a strange and distorted way, supposed to be about P(E|null hypothesis), and so, yes, two experiments with a p-value of 0.05 should add up to somewhere in the vicinity of p < 0.0025, because it's about likelihoods, which do multiply, and not posteriors.

Comment author: jsteinhardt 31 October 2012 04:31:10PM 0 points [-]

While some frequentist methods do use likelihoods, the mapping from likelihood to p-value is non-linear, so multiplying them would still be a mistake, at least as far as I can tell.