1) Really?

Alexander Hamilton appears on how many distinct denominations of US Currency

It's the present tense that throws me. I would expect the question to be 'has appeared'. Whatever.

2) That's not what I meant. I mean, you can turn each individual person's prediction/probability pair into a gaussian curve. It is centered on their answer with width such that a 10 year window contains that much probability. You can then use that to get the probability this distribution - and thus, by proxy, the respondent - assigns to the actual year.

3) On such a small data set you can't get rid of luck, let alone differences in knowledge. I think that by picking out people who got the same number correct you do a pretty good job of de-confounding that. It cuts sideways across the bins in the 'mean % correct' vs 'mean % confidence' graph which showed flat performance across confidence, in a way that you can't do straightforwardly otherwise.