It may be helpful to read some related posts (linked by lukeprog in a comment on this post): Estimate stability, and Model Stability in Intervention Assessment, which comments on Why We Can't Take Expected Value Estimates Literally (Even When They're Unbiased). The first of those motivates the A_p (meta-probability) approach, the second uses it, and the third explains intuitively why it's important in practice.