I'm not sure how much this relates to the question of calibrating self-assessments in particular, but when I was at Yahoo, a few of us made this game -- http://yootles.com/calibration/guessum -- with the objective of being a fun way to train yourself to be better calibrated, at least for mundane estimation tasks. It was based on this article: http://messymatters.com/calibration (and especially the follow-on post with the results, and the speculation that people could learn to be better calibrated, hence the game).