army1987 comments on How to Fix Science - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (141)
Bayesian methods are better in a number of ways, but ignorant people using a better tool won't necessarily get better results. I don't think the net effect of a mass switch to Bayesian methods would be negative, but I do think it'd be very small unless it involved raising the general statistical competence of scientists.
Even when Bayesian methods get so commonplace that they could be used just by pushing a button in SPSS, researchers will still have many tricks at their disposal to skew their conclusions. Not bothering to publish contrary data, only publishing subgroup analyses that show a desired result, ruling out inconvenient data points as "outliers", wilful misinterpretation of past work, failing to correct for doing multiple statistical tests (and this can be an issue with Bayesian t-tests, like those in the Wagenmakers et al. reanalysis lukeprog linked above), and so on.
I think that teaching Bayesian methods would itself raise the general statistical competence of scientists as a side effect, among other things because the meaning of p-values is seriously counter-intuitive (so more scientists would actually grok Bayesian statistics in such a world than actually grok frequentist statistics right now).
You could well be right. I'm pessimistic about this because I remember seeing lots of people at school & university recoiling from any statistical topic more advanced than calculating means and drawing histograms. If they were being taught about conjugate priors & hyperparameters I'd expect them to react as unenthusiastically as if they were being taught about confidence levels and maximum likelihood. But I don't have any rock solid evidence for that hunch.