It may be difficult to find a reasonable Bayesian interpretation, and it may only approximate said interpretation, but if it's at all useful, it will have one.
Observation: This theory that you've stated here --- that any useful frequentist method will have a Bayesian interpretation --- doesn't serve much in the way of controlled anticipation. Because there is so much flexibility in choosing priors and a loss function, the fact that "every useful frequentist method will be a Bayes method in disguise" doesn't tell us much about what frequentist methods will turn out to be useful.
It seems to me that the wisdom to treat beliefs as anticipation controllers is more general, and I think more important, than the choice of Bayesian vs Frequentist inference methods. Each school has their own heuristic for quantifying this wisdom.
As for Bayesian vs Frequentist interpretations of what the word "probability" means, I think that's a different (and sillier) debate.
This theory that you've stated here --- that any useful frequentist method will have a Bayesian interpretation --- doesn't serve much in the way of controlled anticipation.
A frequentist tool only works insomuch as it approximates a Bayesian approach. As such, given the domain in which it works well, you can prove that it approximates the Bayesian answer.
For example, if you're trying to find the probability of a repeatable event ending in success, the frequentist method says to use success/total. The Bayesain approach with a maximum entropy prior gives (...
I recently started watching an interesting lecture by Michael Jordan on Bayesians and frequentists; he's a pretty successful machine learning expert that takes both views in his work. You can watch it here: http://videolectures.net/mlss09uk_jordan_bfway/. I found it interesting because his portrayal of frequentism is much different than the standard portrayal on lesswrong. It isn't about whether probabilities are frequencies or beliefs, it's about trying to get a good model versus trying to get rigorous guarantees of performance in a class of scenarios. So I wonder why the meme on lesswrong is that frequentists think probabilities are frequencies; in practice it seems to be more about how you approach a given problem. In fact, frequentists seem more "rational", as they're willing to use any tool that solves a problem instead of constraining themselves to methods that obey Bayes' rule.
In practice, it seems that while Bayes is the main tool for epistemic rationality, instrumental rationality should oftentimes be frequentist at the top level (with epistemic rationality, guided by Bayes, in turn guiding the specific application of a frequentist algorithm).
For instance, in many cases I should be willing to, once I have a sufficiently constrained search space, try different things until one of the works, without worrying about understanding why the specific thing I did worked (think shooting a basketball, or riffle shuffling a deck of cards). In practice, it seems like epistemic rationality is important for constraining a search space, and after that some sort of online learning algorithm can be applied to find the optimal action from within that search space. Of course, this isn't true when you only get one chance to do something, or extreme precision is required, but this is not often true in everyday life.
The main point of this thread is to raise awareness of the actual distinction between Bayesians and frequentists, and why it's actually reasonable to be both, since it seems like lesswrong is strongly Bayesian and there isn't even a good discussion of the fact that there are other methods out there.