We do ten experiments. A scientist observes the results, constructs a theory consistent with them, and uses it to predict the results of the next ten. We do them and the results fit his predictions. A second scientist now constructs a theory consistent with the results of all twenty experiments.
The two theories give different predictions for the next experiment. Which do we believe? Why?
One of the commenters links to Overcoming Bias, but as of 11PM on Sep 28th, David's blog's time, no one has given the exact answer that I would have given. It's interesting that a question so basic has received so many answers.
I wrote:
To the extent that predictions 11-20 and 21 are generated by different independent "parts" of the theory, the quality of the former part is evidence about the quality of the latter part via the theorist's competence.
...however, this is much less true of cases like Newton or GR where you can't change a small part of the theory without changing all the predictions, than it is of cases like "evolution theory is true and by the way general relativity is also true", which is really two theories, or cases like "Newton is true on weekdays and GR on weekends", which is a bad theory.
So I think that to first order, Peter's answer is still right; and moreover, I think it can be restated from Occam to Bayes as follows:
Experiments 11-20 have given the late theorizer more information on what false theories are consistent with the evidence, but they have not given the early theorizer any usable information on what false theories are consistent with the evidence. Experiments 11-20 have also given the late theorizer more information on what theories are consistent with the evidence, but this does not help the late theorizer relative to the early theorizer, whose theory after all turned out to be consistent with the evidence. So experiments 11-20 made it more likely for a random false late theory to be consistent with the evidence, relative to a random false early theory; but they did not make it more likely for a random late theory to be consistent with the evidence, relative to the early theory that was put forward. Therefore, according to some Bayes math that I'm too lazy to do, it must be the case that there are more false theories among late theories consistent with the evidence, than among early theories consistent with the evidence.
Does this make sense? I think I will let it stand as my final answer, with the caveat about theories with independent parts predicting different experiments, in which case our new information about the theorists matters.