Since the understanding-the-question module is similar for each person, though, the actual errors aren't evenly distributed across questions, so they will underestimate on "easy" questions and overestimate on "hard" ones, if easy and hard are determined afterwards by percentage that get the answer correct.

That seems reasonable to me, yes, as an easy way for a question to be 'hard' is if most answerers interpret it differently from the questioner.