A: first theory
B: second theory
D: data accumulated between the 10th and 20th trials
We're interested in the ratio P(A|D)/P(B|D).
By Bayes' Theorem:
P(A|D) = P(D|A)P(A)/P(D)
P(B|D) = P(D|B)P(B)/P(D)
Then
P(A|D)/P(B|D) = P(D|A)P(A)/(P(D|B)P(B)).
If each theory predicts the data observed with equal likelihood (that is, under neither theory is the data more likely to be observed), then P(D|A) = P(D|B) so we can simplify,
P(A|D)/P(B|D) = P(A)/P(B) >> 1
given that presumably theory A was a much more plausible prior hypothesis than theory B. Accordingly we have P(A|D) >> P(B|D), so we should prefer the first theory.
In practice, we these assumptions may not be warranted. In which case, we have to balance the likelihood of the priors (as we can best guess) and how well the theories predict the observed data (as we should be able to estimate directly from the theories).
My approach: (using Bayes' Theorem explicitly)
A: first theory
B: second theory
D: data accumulated between the 10th and 20th trials
We're interested in the ratio P(A|D)/P(B|D).
By Bayes' Theorem:
P(A|D) = P(D|A)P(A)/P(D)
P(B|D) = P(D|B)P(B)/P(D)
Then
P(A|D)/P(B|D) = P(D|A)P(A)/(P(D|B)P(B)).
If each theory predicts the data observed with equal likelihood (that is, under neither theory is the data more likely to be observed), then P(D|A) = P(D|B) so we can simplify,
P(A|D)/P(B|D) = P(A)/P(B) >> 1
given that presumably theory A was a much more plausible prior hypothesis than theory B. Accordingly we have P(A|D) >> P(B|D), so we should prefer the first theory.
In practice, we these assumptions may not be warranted. In which case, we have to balance the likelihood of the priors (as we can best guess) and how well the theories predict the observed data (as we should be able to estimate directly from the theories).