I really appreciate your thoughts on this thread :-)

I think that on any given specific question, it's generally possible for a sufficiently intelligent and determined person to beat out elite conventional wisdom. To the extent that you and I disagree, the point of disagreement is how much investigation one has to do in order to overturn elite conventional wisdom. This requires a subtle judgment call.

To give an example where not investigating in sufficient depth leads to problems, consider the error that GiveWell found in the DCP-2 cost-effectiveness estimate for deworming. A priori one could look at the DCP-2 and say "Conventional wisdom among global health organizations is to do lots of health interventions, but they should be trying to optimize cost-effectiveness, deworming is the most cost-effective intervention, there are funding gaps for deworming, so the conventional wisdom is wrong." Such a view would have given too little weight to alternative hypotheses (e.g. it being common knowledge among experts that the DCP-2 report is sloppy, experts having knowledge of the low robustness of cost-effectiveness estimates in philanthropy, etc.)