If you don't understand why that is so, read the articles about the t-test and the F-test. The tests compute what a difference in magnitude of response such that, 95% of the time, if the measured effect difference is that large, the null hypothesis (that the responses of all subjects in both groups were drawn from the same distribution) is false.

No, the correct form is: * The tests compute a difference in magnitude of response such that if the null hypothesis is true, then 95% of the time the measured effect is not that large. The form you quoted is a deadly undergraduate mistake.

I read through most of the comments and was surprised that so little was made of this. Thanks, VincentYu. For anyone who could use a more general wording, it's the difference between:

• P(E≥S|H) the probability P of the evidence E being at least as extreme as test statistic S assuming the hypothesis H is true, and

• P(H|E) the probability P of the hypothesis H being true given the evidence E.