On Will Newsome's IRC channel someone mentioned the idea that you could totally automate the ITT into a mass-league game with elo ratings and everything (assuming there was some way to verify true beliefs at the beginning.)

Forget trying to use people's actual beliefs anywhere in the process; it's simpler just to let people play the ITT for a lot of disjoint positions, so that they only get the bonus for their actual beliefs at most once. This mildly penalizes people with extremely idiosyncratic beliefs, but such people wouldn't even be able to play the current ITT.