I think a lot about signal detection theory, and I think that's still the best I can come up with for this question. There are false positives, there are false negatives, they are both important to keep in mind, the cost of reducing one is an increase in the other, humans and human systems will always have both.

So, for example, even the most over-generous public welfare system will have deserving people off the dole and even the most stingy system will have undeserving recipients (by whatever definition), so the question (for a welfare system, say) isn't how do we prevent abuse, but how many abusers are we willing to tolerate for every 100 deserving recipients we reject? Also useful in lots of medical discussions, legal discussions, pop science discussions, etc.

Discussing "humility" and "arrogance" is difficult without careful definitions. I was thinking about this recently; this is how I would like to define them.

Whenever I end up feeling like I have been arrogant, it is because I underestimated someone else's abilities, and I ended up surprised by what they were capable of. If humility is the opposite of arrogance, then humility is the ability to accept, as your prior, that somebody you meet just might end up being more wise or more accomplished than you. To be arrogant is to fail to realize that you might have something to learn from other people.

(These definitions of arrogance and humility thus only relate to mental habits, not to social behaviors.)

Note how this makes humility valuable -- if you expect everyone around you to be dumb and inferior and not worth learning from, if you don't give others the chance to prove you otherwise, you're going to miss out on everything that you could be learning from them. I wouldn't expect your putative arrogant academic to have very many fruitful collaborations.

So yes, I would say that arrogance is bad intellectual hygiene -- it's having the wrong priors about the people around you.

Note also that it's also possible to be unfair to oneself in this way. Impostor syndrome should not be confused for humility. High self-esteem should not be confused for arrogance.

... I realize only after writing all of this that there's also intellectual arrogance and intellectual humility; it seems that they can be modeled the same way, but with ideas instead of people.

I believe you're confusing arrogance and closedmindedness. There is a correlation between openmindedness and intelligence. I do believe openmindedness is a form of rationality. I'm not sure how good rationality training is at increasing it. Personality Psychology indicates Openness to Experience is a fairly stable trait; not one that can be taught.

By "arrogance" do you mean overestimating your abilities, or just being annoying about it? The first one is problematic, but so is underestimating. You should calibrate yourself and get an accurate view of your abilities. The second one seems to be more about some kind of signalling. It has an obvious disadvantage of alienating people, but it must have some kind of advantage or people wouldn't do it.

This work articulates an attack on the use of conjugate priors in a Bayesian analysis: http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.ba/1340369826 In their words, "conjugate priors may lead to a dogmatic analysis."

Sorry for necro.

Why do you care which scientific theories are right?

I want to be good at being wrong

Not 'less' wrong? I'm not even sure from reading the rest of your comment whether this is a typo or whether you intended something I didn't understand.