I agree that this kind of outside view argument doesn't provide absolute certainty. However, it does provide evidence that part of your reasons for believing X are irrational reasons that you're rationalizing. Reduce your probability estimate of X accordingly.

I should also take this occasion as a reminder to think twice the next time I'm inclined to claim that I'm paraphrasing something fairly and yet in such a way that it still sounds silly;

Note, that the formulation presented here is one I came up with on my own while searching for the bayesstructure behind arguments based on the outside view.