Haven't posted in quite a while.
Suppose you have a big, complicated question that you're not sure of the answer to, and you want to seek an adviser to guide you. One kind of adviser is someone whose opinion, by your lights, constitutes strong evidence regarding the answer; on the basis of that opinion alone you are prepared to substantially update your beliefs. Of course you may profit from further discussion beyond just hearing the adviser's opinion on the big question: since the question is complicated, hearing his or her reasoning or evidence on different elements of the big question may be valuable, but the point is that there are some advisers for whom just knowing their ultimate judgment moves the needle a lot for you. Such people might be termed "good guides."
But there may be other potential advisers whose ultimate opinion on the big question you don't credit much at all, but who you think might still have valuable insight into some important element of the question. A good example for me is "Chicago School" Industrial Organization Economics. It's members had some insights that are absolutely true and important ("one monopoly profit" and related ideas), and that the people who I would have regarded as my "good guides" had I been around at the time did not have before them. No analyst who does not understand those insights can be a good analyst, and no analysis that ignores them can be correct. But simply knowing what an orthodox Chicago School economist thinks about some big question would move me very little. They are a valuable part of my "intellectual portfolio" (to use a phrase favored by Brad DeLong) and I would be a fool to dismiss them. But they are only providers of valuable input, not good guides.
I think the distinction between these two types of advisers is often missed. If you believe my example (if not, substitute one of your own, the point of this post is not to debate IO), there are a bunch of expert economists (Chicago School types) who should have fancy prestigious professorships, and whose arguments should be given careful consideration; and there are another bunch of expert economists who should have fancy prestigious professorships, whose arguments should be given careful consideration, and whose advice should be heeded. Leave aside the practical difficulty of knowing which is which if you are, say, a reporter or a policy-maker. The point is that there should be two buckets for two different types of prestigious advice-giver, but we only really have one.
You have. To quote from your article:
When I boil down your post+comment and look for sense, what I get is something prosaic like 'some people are so expert in a narrow area that their opinion alone is enough for me, but otherwise aren't very good on average; other people are that expert, but in a broader area'.
The point is that these are two very different kinds of valuable advisers, but the distinction is often missed.
And while I do think in real life there is something of a dichotomy between "people whose final judgment I trust on questions like X," and "people whose final judgment I don't trust but who I still want to hear what they have to say," I think a similar point could be made with more than two categories.