I have spent some time extracting bets and predictions from Long Bets under my account.
So far I have all open bets with fixed dates imported, and roughly a third of the predictions.
It would be nice if a bunch of LWers could go in and put down their own probabilities. It's true that most of them don't come due any time soon - looking at upcoming predictions I see the first Long Bet item coming up in 5 months, followed by 2 or 3 within half a year after that. But the more who use it, the more short-term predictions and the more useful. The rich get richer, etc.
I've finished importing all the sensible bets and predictions from LB. I suppose the next target is Wrong Tomorrow, and then I'll turn to Intrade.
(I see next to no contributions from LWers. This disappoints me; do we all think we are well-calibrated or what?)
Our hosts at Tricycle Developments have created PredictionBook.com, which lets you make predictions and then track your calibration - see whether things you assigned a 70% probability happen 7 times out of 10.
The major challenge with a tool like this is (a) coming up with good short-term predictions to track (b) maintaining your will to keep on tracking yourself even if the results are discouraging, as they probably will be.
I think the main motivation to actually use it, would be rationalists challenging each other to put a prediction on the record and track the results - I'm going to try to remember to do this the next time Michael Vassar says "X%" and I assign a different probability. (Vassar would have won quite a few points for his superior predictions of Singularity Summit 2009 attendance - I was pessimistic, Vassar was accurate.)