You need to work with probabilities, and then make statements about your expected Bayes-score instead of truth or falsity; then you'll be consistent. I have a post on this but I can't remember what it's called.