I suppose I should have been clear that by cold fusion in this context we were explicitly discussing the Pons-Fleischmann sort of set-up. I don't actually think that Pons and Fleischmann were frauds. I'm more inclined to believe that it was an example of poor controls and wishful thinking. I'm aware of other types of low-energy fusion such as fusors and polywells but know that I don't have anywhere near the expertise to evaluate them.
In any event, your first paragraph strongly argues strongly that I should discount my confidence estimates for other fields much more than I do.
One issue that I've noticed in discussions on Less Wrong is that I'm much less certain about the likely answers to specific questions than some other people on Less Wrong. But the questions where this seems to be most pronounced are mathematical questions that are close to my area of expertise (such as whether P = NP). In areas outside my expertise, my apparent confidence is apparently often higher. Thus, for example at a recent LW meet-up I expressed a much lower probability estimate that cold fusion is real than what others in the conversation estimated. This suggests that I may be systematically overestimating my confidence in areas that I don't study as much, essentially a variant of the Dunning-Krueger effect. Have other people here experienced the same pattern with their own confidence estimates?