the questions where this seems to be most pronounced are mathematical questions that are close to my area of expertise (such as whether P = NP)
On a tangential note, exactly how close is this to your area of expertise? In my experience it tends to be mathematicians who are in related areas but don't actually work on complexity theory directly who insist on being agnostic about P vs NP - almost all complexity theorists are pretty much completely convinced of even stronger complexity theoretic assumptions (eg, I bet Scott Aaronson would give pretty good odds on BQP != NP).
I'm not entirely sure how tangential this is, as it seems to suggest that there may be some sort of sweet point of expertise (at least on this question) - any layman would take my word for it that P != NP, most non-CS mathematicians would refuse to have an opinion and most complexity theorists are convinced of its truth for their own reasons. I guess this might be something that's unique to mathematics, with its insistence on formal proof as a standard of truth, can anyone thing of anything similar in other fields?
Not "almost all are completely convinced"; according to this poll, 61 supposed experts "thought P != NP" (which does not imply that they would bet their house on it), 9 thought the opposite and 22 offered no opinion (the author writes that he asked "theorists", partly people he knew, but also partly by posting to mailing lists - I'm pretty sure he filtered out the crackpots and that enough of the rest are really people working in the area)
Even that case wouldn't increase the likelyhood of P != NP to 1-epsilon, as experts ha
One issue that I've noticed in discussions on Less Wrong is that I'm much less certain about the likely answers to specific questions than some other people on Less Wrong. But the questions where this seems to be most pronounced are mathematical questions that are close to my area of expertise (such as whether P = NP). In areas outside my expertise, my apparent confidence is apparently often higher. Thus, for example at a recent LW meet-up I expressed a much lower probability estimate that cold fusion is real than what others in the conversation estimated. This suggests that I may be systematically overestimating my confidence in areas that I don't study as much, essentially a variant of the Dunning-Krueger effect. Have other people here experienced the same pattern with their own confidence estimates?