This is a linkpost for http://putanumonit.com/2016/12/18/multiplicitous
Thank you for writing that! To sum up:
Alice: I got this result with p<0.05
Bob: How many hypotheses did you test?
Alice: 10
Bob: Then you got the result with p<0.5
This is going straight into my toolbox.
maybe WE SHOULD STOP USING FUCKING P-VALUES FOR HYPOTHESIS TESTING.
There's some... tension between this correct statement and the rest of your blog post :-/
You're right, I added a paragraph explaining why I do this:
P-values aren’t very useful in getting close to the truth, but they’re everywhere, they’re easy to work with and they’re moderately useful for getting away from bullshit. Since the latter is our goal in this essay we’ll stick with looking at p-values for now.
Since you see p-values reported all the time and they're easy to calculate (because they're uniformly distributed), there's value in extracting whatever useful evidence there is in p-values.
Unfortunately for the author's "Rule of two Ns", "Bonferonni" is actually spelt "Bonferroni".
You could correct the post as "Both 'Dunn' and 'Bonferroni' have exactly two 'n' in them. Thus we have etc."
That's a good suggestion, but I actually had to update that entire section because the math had an embarrassing mistake and I renamed the rule.
By the way, I have a policy on my blog of paying bounties to commenters who catch major errors. I sent $5 to the commenter who pointed out where I messed up the math. If you see a really stupid mistake on Putanumonit, leave a comment there and make a buck.