I have found, strangely enough, that switching from percentages (or values between 0 and 1) to something that feels more "discrete", like a scale of 0 to 10 or a "five-star" system, sometimes helps with this.

This makes perfect sense to me. I feel far more comfortable converting my sense of credibility to an intensity scale of 1 to 100 than converting those intensities to probabilities.