Yes, that's what I meant by "very rare:" there are situations where it happens, like the model that you gave, but I don't think ones that happen in real life likely to contribute a very large effect. You need really insane publication bias to get a large effect there.
http://fivethirtyeight.com/features/a-formula-for-decoding-health-news/
Their actual implementation is rather hand wavy, but I think getting people pointed in the right direction is a lot more important than the particulars. The high visibility of FiveThirtyEight makes me think this is a big win for sanity. I'm curious as to what others think. Which parts were done well? What might have been done differently?