I think the problem here is with many trivia questions you either know the answer or you don't; the dominant factor in my results so far is that I either have no answer in mind, assign 0 probability to my being right and am correctly calibrated there, and then all of my answers at other levels of certainty have turned out right so far so my calibration curve looks almost rectangular.

I might just be getting accurate information that I'm drastically underconfident, but I think this might be one of the worse types of questions to calibrate on. I mean, even if the problem is just that I'm drastically underconfident on trivia questions and shouldn't be assigning less than 50% probability to any of my answers when I have an answer, that sounds sufficiently unrepresentative of most areas where you need calibration, and how most people perform on other calibration tests, for this to be a pretty bad measure of calibration.

Perhaps it would be better as a multiple choice test, so one can have possible answers raised to attention that may or may not be right, and assign probabilities to those?