My ideas for the interaction are not well formed. I'm more interested in what I could data mine. I imagine something like predictionbook, but miniaturized to daily situations. Maybe it could be a little like a game. You have a circle of friends and you make predictions about things. You're all at a restaurant together. Someone says they think the U.S. dollar has gone down in real value since leaving the gold standard. You use an iPhone app to state the claim and then people in your circle can add their predictions or something, and maybe someone can accept an answer in a way that's similar to stack overflow. It might require some honor system for your peers to up-vote you when you get it right and down-vote you when you get it wrong.
But the valuable part for me would be to see a huge time series of my rights and wrongs, broken down by some text analysis of the statements of the claims, and possibly linked to confidence ratings in my answers. If I had a year's worth of data from this app, on questions ranging from "What is Obama's voting record on bills involving the accessibility of birth control?" to "Will Kentucky win the basketball championship?", I think I could gain a lot by seeing how my overconfidence breaks down by subject area, which topics I am less willing to say oops and change my mind about, which areas I am more inclined to guess correctly, and so on.
It sounds like a fun project. A version of stackoverflow where you decide which circles of friends can be part of your "honor system" by fact checking your claims. Presumably making it a Facebook app that posts twitter-style synopses of the predictions you want to make public would be a good interface for it, and then figuring out what kind of textual machine learning algorithms need to go underneath would be a bit harder.
We know politics makes us stupid, but now there's evidence (pdf) that politics makes us less likely to consider things from another's point of view. From the abstract:
As Will Wilkinson notes:
Now this isn't necessarily a negative effect - you might argue that it's bias correcting. But implicitly viewing them as so different that it's not even worth thinking about things from their perspective is scary in itself.