Surely you can teach yourself to compare intuitive certainty to probabilities, though. I mean, if you come up with rough labels for levels of intuitive certainty, and record how often each label is right or wrong, you'd get a really rough corresponding probability already.

Edit: Oh, this is predictionbook's raison d'être.