I realize Eliezer holds great sway on this blog, but I think people here ought to question a bit more closely some of his most winning arguments in favor of casting out frequents for Bayesianism. I've only read this blog around 4 times, and each time I've found a howler apparently accepted. But putting those aside, I find it curious that the results on psychological biases that is given so much weight on this blog are arrived at and affirmed by means of error statistical methodology. error statistics.com
Frequentism is as abused as "orthodox statistics", and in any event, tends to evoke a conception of people interested in direct inference: assigning a probability (based on observed relative frequencies) to outcomes. Frequentism in statistical inference, instead, refers to the use of error probabilities--based on sampling distributions-- in order to assess and control a method's capability to probe a given discrepancy or inferential flaw of interest. Thus, a more suitable name would be error probability statistics, or just error statistics. One infers, for example, that a statistical hypothesis or other claim is well warranted or severely tested just to the extent that the method was highly capable of detecting the flaw, and yet routinely produces results indicating the absence of a flaw. But the most central role of statistical method in the error statistical philosophy is to block inferences on a variety of grounds, e.g., that the method had little capacity to distinguish between various factors, biases, failing to give the assumptions of the models used a sufficiently hard time.
But the real reason I wrote is because the first few sentences of this post made me think that perhaps the professor was me! I'm glad to hear there are other female philosophers of science who are frequentists. yet it wasn't me, given the rest of the post.
I'm sorry to see such wrongheaded views of frequentism here. Frequentists also assign probabilities to events where the probabilistic introduction is entirely based on limited information rather than a literal randomly generated phenomenon. If Fisher or Neyman was ever actually read by people purporting to understand frequentist/Bayesian issues, they'd have a radically different idea. Readers to this blog should take it upon themselves to check out some of the vast oversimplifications... And I'm sorry but Reichenbach's frequentism has very little to do with frequentist statistics--. Reichenbach, a philosopher, had an idea that propositions had frequentist probabilities. So scientific hypotheses--which would not be assigned probabilities by frequentist statisticians--could have frequentist probabilities for Reichenbach, even though he didn't think we knew enough yet to judge them. He thought at some point we'd be able to judge of a hypothesis of a type how frequently hypothesis like it would be true. I think it's a problematic idea, but my point was just to illustrate that some large items are being misrepresented here, and people sold a wrongheaded view. Just in case anyone cares. Sorry to interrupt the conversation (errorstatistics.com)
If there was a genuine philosophy of science illumination it would be clear that, despite the shortcomings of the logical empiricist setting in which Popper found himself , there is much more of value in a sophisticated Popperian methodological falsificationism than in Bayesianism. If scientists were interested in the most probable hypotheses, they would stay as close to the data as possible. But in fact they want interesting, informative, risky theories and genuine explanations. This goes against the Bayesian probabilist ideal. Moreover, you cannot falsify with Bayes theorem, so you'd have to start out with an exhaustive set of hypotheses that could account for data (already silly), and then you'd never get rid of them---they could only be probabilistically disconfirmed.
No, the multiple comparisons problem, like optional stopping, and other selection effects that alter error probabilities are a much greater problem in Bayesian statistics because they regard error probabilities and the sampling distributions on which they are based as irrelevant to inference, once the data are in hand. That is a consequence of the likelihood principle (which follows from inference by Bayes theorem). I find it interesting that this blog takes a great interest in human biases, but guess what methodology is relied upon to provide evidence of those biases? Frequentist methods.
Y'all are/were having a better discussion here than we've had on my blog for a while....came across by chance. Corey understands error statistics.
Just a couple of points on this discussion, which I'm sure I walked in at the middle of: (1) One thing it illustrates is the important difference between what one "should" believe in the sense of it being prudential in some way, versus a very different notion: what has or has not been sufficiently well probed to regard as warranted (e.g., as a solution to a problem, broadly conceived). Of course, if the problem happens to be "to promote luckiness", a well-tested solution could turn out to be "don't demand well-testedness, but think on the bright side."
(2) What I think is missing from some of this discussion is the importance of authenticity. Keeping up with contacts, and all the other behaviors, if performed as part of a contrived plan will backfire.