These posts at Slate Star Codex seem relevant:

• Beware The Man Of One Study

  But I worry that most smart people have not learned that a list of dozens of studies, several meta-analyses, hundreds of experts, and expert surveys showing almost all academics support your thesis – can still be bullshit.

• The Control Group Is Out Of Control

  How do I know it’s crap? Well, I use my personal judgment. How do I know my personal judgment is right? Well, a smart well-credentialed person like James Coyne agrees with me. How do I know James Coyne is smart? I can think of lots of cases where he’s been right before. How do I know those count? Well, John Ioannides has published a lot of studies analyzing the problems with science, and confirmed that cases like the ones Coyne talks about are pretty common. Why can I believe Ioannides’ studies? Well, there have been good meta-analyses of them. But how do I know if those meta-analyses are crap or not? Well…

Have you researched both sides so thoroughly that you consider yourself equal to or better than the opposing experts? If so, to what do you attribute the mistakes of your counterparts? Have you carefully considered the possibility that you are the one who's mistaken?

In many cases, yes - I do consider myself equal to or better than the opposing experts. I also think long and hard about being the one who's wrong, and I have changed my mind on many topics as a result.

In a number of cases which are not particularly political, I can attribute the mistakes of my of my counterparts to lack of understanding in the field.

However, in the other cases, I can also point at a deeper, more troubling concern: an unwillingness to use computable functions, a willingness to use 'incomparables' or infinite value weightings. Risk to human life from nuclear waste can be computed in dollar terms, yet activists refuse to do so. Similarly, value of a fetus can also be computed in dollar terms, but such is never done. In my eyes, this is a clear and obvious mistake.

Do you rely on the majority of experts?

In many cases, yes. By looking at the bulk of a field, neighboring related fields, advocates and detractors, you can get a good feel for a topic:

• Human-induced climate change has the vast bulk of experts advocating for it, with a rather small minority of detractors, most of whom are below average for the expert pool.

• Anti-evolution creationists have effectively no experts in their corner.

• The vast majority of expert economists look at economic crashes through the lens of 'this is the most complicated construct mankind has ever created'. The minorities of economists who loudly advocate for singular failure cases and singular fixes are not only in the minority, but often conflict with each other in dramatic ways.

In cases where it appears that the majority is wrong, I've generally found that it's because the majority doesn't actually consist of experts on the specific topic. (See below in regard to doctors and molecular biology.) There are also situations where the pool of experts is quite small indeed, for example medical doctors with physics/nanotechnology backgrounds, where a more general pool would produce the wrong consensus.

Do you think that there are powerful arguments that simply haven't been addressed by the other side? To what do you attribute the fact that these arguments haven't been addressed?

Sometimes, yes. An excellent example is that the vast majority of doctors do not believe that aging is reversible and the diseases of old age are curable. This largely appears to be because most doctors are not molecular biologists, and do not really have the skills to evaluate what is actually possible in molecular biology (and what is expected to become possible in the next few decades.)

In this case, the 'experts' are from related fields, but are not as expert on the specific topic as would be expected.

Do you just remain more or less an agnostic on every controversial subject?

For all the subjects I have fairly strong opinions on, there are a lot which I don't because I lack the expertise. Even in some areas where I would be expected to have an opinion, I don't - I am a software engineer, yet I am divided on hard versus soft takeoff of AI. I lean toward soft takeoff for various reasons, but hard takeoff is definitely in the 'plausible' category.

Or do you perhaps admit that ultimately your beliefs are at least partially founded on non-rational reasons?

Absolutely. One of our goals should be to clean up instances of this as they are discovered, and attempt to change protocols to limit it in the future.

Look at past predictions by the "experts" and compare them to what actually happened. If the experts are proposing courses of action did their previous proposals work? What is the field's analog of "I believe physicists know what they're talking about because airplanes fly"?

I think there are several levels to this question:

1. Any issue based on Politics or Religion will have experts on both sides arguing for their case based on their own personal choice and biases - these are more often than not, arguments and discussions rooted entirely on opinion.

2. For controversial issues in Science and Economics I will try to determine if there are any biases based on point 1. In some cases this becomes visible based on the political leanings of the Scientists and / or corporate sponsorship of the study.

3. In the Maths world there doesn't seem to be any controversial issues over the results (which is why I kind of like maths a lot!), however there are many issues on which approach to take and how useful a result is. For those sorts of predictive issues I don't usually have a strong opinion

So in general, it comes down to what the biases are. Everyone has an opinion, and these are (or ought to be) less relevant the deeper you get into the Science and Maths.

There is a controversy in the field of nature conservation which boils down to 'what amount (& other specifics) of human intervention can be allowed in a habitat, such that it is still considered 'wild'?'. The question can sometimes be viewed as a set of 'decision - human intervention - outcome - monitoring', and this is a practical approach with classifiable results, BUT it is only applicable to certain types of habitats, usually (in Continental Europe, I think) already quite anthropogenically transformed. (There are other restrictions, too.) So when you have to decide for any particular patch of 'wilderness' the best way to preserve it, you side with the experts who say 'touch nothing and let only Nature rule there', and the number of species in that place goes down (because there is not enough large mammals, etc.) OR you side with those who say 'traditional land use!' In this case, you get a nice, heterogeneous reserve and a positive feedback loop that will eat it sooner or later. Damned if you do, damned if you don't.

I try not to pick sides, unless I know the habitat well, but whatever I do, I seem to lose.

First, I ask myself "what is the likelihood this belief necessitate actions which may cause harm that cannot be undone?" The higher the likelihood the less I am drawn to adopt the belief. Some religious and political thinkers hold beliefs that they say mandate killing people, for example.

Second, I ask myself "how much do I need to hold a belief in this area at all?" The lower the need the less I am drawn to adopt the belief. I am not a Hindu or a Republican and it matters to me not at all if a particular belief is for or against my being a Hindu or a Republican.

Third, I ask "will I regret not trying this belief more than I regret trying it?" Temporarily adoping beliefs to stress-test them has yielded fantastic positive results and cost only the shedding prior false or incomplete beliefs.

None of these questions are often asked in such stark terms but they are in my mind in some way.

I wrote as a little part of a comment in the middle of a longish thread:

There is a paper "Experts: Which ones should you trust" addressing this issue by Alvin Goldman -- you need google scholar or JSTOR or something to actually get the article), one of the biggest names in epistemology and specifically social epistemology. Actually I don't think the article does very much to resolve the issue unfortunately.

One article (cited in Goldman "Experts...") that I really like is John Hardwig "The Role of Trust in Knowledge", which gets at the critical need for experts to trust other experts, and illustrates with examples of scientific and mathematical accomplishment that just don't fit in one person's head.

Besides the qualities of individuals, we must ask whether a particular study discipline has anything trustworthy to say. As Dustin said:

My goal ... on subjects on which there truly is not a expert consensus is to acknowledge that there is no consensus and thus not choose one side or another.

DanArmak spoke of the "consensus-making process in the field" which "has to be explicitly rational and truth-seeking" -- a very good point, I think, that I've tried to illustrate in "Global Warming and the Controversy: What is Scientific Consensus? Continental Drift as Example".

One point: An area of scientific study has to be tractable (this is relative to available technology -- medical science remained largely intractable until pretty recently), and there has to be a there there. See cartoon: http://xkcd.com/451/

I got the impression from some passing reference (if I didn't imagine it) that Stephen Toulmin has had some things to say about this aspect of "what makes (a) science work" (still haven't found something he wrote to confirm this); I've made some attempts at dealing with it myself in "What is A Machine? Natural Machines and Origins of Science" and "Finding Your Invisible Elephant. A Science Requires, and is Shaped by, a Tractable Subject Matter". The articles aren't as polished as I wish they were - my New Years resolution is to do better.

Most of my beliefs* on controversial issues stem from one of the following:

• I voluntarily exposed myself to persuasive media (political magazines, fiction novels, this website) for entertainment purposes.
• A certain position is in my self-interest. (I oppose using physically painful aversive stimuli as an autism treatment without informed consent directly from the patient, not only because I conveniently have a moral system that opposes them, but also because I do not want to be electrically shocked without my consent.)
• I want to be a member of a political coalition, because they are working towards one of my goals. They also have some other goals that I didn't originally care about either way, but membership in the coalition required that I self-modify to care more about those goals, so I did so by reading lots of persuasive arguments online until I valued their goal at least slightly.

*The "beliefs" referenced don't really meet the LW definition of belief, and in terms of local concepts, are much closer to utility function differences. My coalition agrees with other groups about what the consequences of [action X] will be, but disagrees about the moral value of those consequences. A lot of members of my coalition make arguments to group A that are equivalent to arguing "You should vote this way, because then more paperclips will exist" to humans. For example, my coalition could have blamed [Organization C] for working with [near-universally politically toxic Group D], but instead they complain about how [Organization C] portrays the world as corresponding to my coalition's opponents' beliefs in their advertising materials. This will not persuade anyone who does not already accept their entire argument.

So, what are your meta-arguments for going with one side or the other for any given controversial subject on which you have an opinion?

If you think that the issue is too close to decisively call one way or the other, it would make sense to lobby for the position with the least backing, in order to provide more balance to the argument.

Most of the really controversial issues only have 2 sides because any nuanced position you have on the topic will have a side selected by the mob. You don't really get to pick a side.