I wrote as a little part of a comment in the middle of a longish thread:

There is a paper "Experts: Which ones should you trust" addressing this issue by Alvin Goldman -- you need google scholar or JSTOR or something to actually get the article), one of the biggest names in epistemology and specifically social epistemology. Actually I don't think the article does very much to resolve the issue unfortunately.

One article (cited in Goldman "Experts...") that I really like is John Hardwig "The Role of Trust in Knowledge", which gets at the critical need for experts to trust other experts, and illustrates with examples of scientific and mathematical accomplishment that just don't fit in one person's head.

Besides the qualities of individuals, we must ask whether a particular study discipline has anything trustworthy to say. As Dustin said:

My goal ... on subjects on which there truly is not a expert consensus is to acknowledge that there is no consensus and thus not choose one side or another.

DanArmak spoke of the "consensus-making process in the field" which "has to be explicitly rational and truth-seeking" -- a very good point, I think, that I've tried to illustrate in "Global Warming and the Controversy: What is Scientific Consensus? Continental Drift as Example".

One point: An area of scientific study has to be tractable (this is relative to available technology -- medical science remained largely intractable until pretty recently), and there has to be a there there. See cartoon: http://xkcd.com/451/

I got the impression from some passing reference (if I didn't imagine it) that Stephen Toulmin has had some things to say about this aspect of "what makes (a) science work" (still haven't found something he wrote to confirm this); I've made some attempts at dealing with it myself in "What is A Machine? Natural Machines and Origins of Science" and "Finding Your Invisible Elephant. A Science Requires, and is Shaped by, a Tractable Subject Matter". The articles aren't as polished as I wish they were - my New Years resolution is to do better.