The problem with "I don't know" is that sometimes you have to make decisions. How do you propose to make decisions if you don't know some relevant mathematical fact X?

Then you need to fully accept that you will be inconsistent sometimes. And compartmentalize your belief system accordingly, or otherwise find a way to deal with these inconsistencies.