It's certainly not the case that Bayesian methods are universally better than frequentist ones.
Examples where frequentist methods are better?
My guess is in hugely overdetermined cases where prior gets swamped by likelihood, and in cases where explicitly representing uncertainty is utterly intractable (like numerical methods), but I'd like to hear it from someone who knows what they are talking about.
Also, if it's not "Bayesian", is there a term for the statistical methodology that is always best in all situations (in the spirit of "rationalists should win")? It seems to me that given that Bayesianism is correct in the ideal sense, the "best" method will always just be the best approximation of the Bayesian answer (where "best" includes factors like computational simplicity).
Well, the most commonly used statistical methods are probably:
All of these are frequentist: logistic regression is quite explicitly computing the maximum-likelihood-estimate of a parameter vector, SVMs are minimizing a surrogate to generalization error, and PCA is a bit weird but is basically just trying to find a low-rank approximation to the data.
ETA: And to answer your other question, I think that would just be called "the best method"; why would we need another name?...
Question in title.
This is obviously subjective, but I figure there ought to be some "go-to" paper. Maybe I've even seen it once, but can't find it now and I don't know if there's anything better.
Links to multiple papers with different focus would be welcome. For my current purpose I have a preference for one that aims low and isn't too long.