Nassim Taleb recently posted this mathematical draft of election forecasting refinement to his Twitter.
(At the more local level this isn’t always true, due to issues such as incumbent advantage, local party domination, strategic funding choices, and various other issues. The point though is that when those frictions are ameliorated due to the importance of the presidency, we find ourselves in a scenario where the equilibrium tends to be elections very close to 50-50.)
So back to the mechanism of the model, Taleb imposes a no-arbitrage condition (borrowed from options pricing) to impose time-varying consistency on the Brier score. This is a similar concept to financial options, where you can go bankrupt or make money even before the final event. In Taleb's world, if a guy like Nate Silver is creating forecasts that are varying largely over time prior to the election, this suggests he hasn't put any time dynamic constraints on his model.
The math is based on assumptions though that with high uncertainty, far out from the election, the best forecast is 50-50. This set of assumptions would have to be empirically tested. Still, stepping aside from the math, it does feel intuitive that an election forecast with high variation a year away from the event is not worth relying on, that sticking closer to 50-50 would offer a better full-sample Brier score.
I'm not familiar enough in the practical modelling to say whether this is feasible. Sometime the ideal models are too hard to estimate.
I'm interested in hearing any thoughts on this from people who are familiar with forecasting or have an interest in the modelling behind it.
I also have a specific question to tie this back to a rationality based framework: When you read Silver (or your preferred reputable election forecaster, I like Andrew Gelman) post their forecasts prior to the election, do you accept them as equal or better than any estimate you could come up with? Or do you do a mental adjustment or discounting based on some factor you think they've left out? Whether it's prediction market variations, or adjustments based on perceiving changes in nationalism or politician specific skills (e.g. Scott Adams claimed to be able to predict that Trump would persuade everyone to vote for him. While it's tempting to write him off as a pundit charlatan, or claim he doesn't have sufficient proof, we also can't prove his model was wrong either.) I'm interested in learning the reasons we may disagree or be reasonably skeptical of polls, knowing it of course must be tested to know the true answer.
This is my first LW discussion post -- open to freedback on how it could be improved
I want to be careful in how I talk about Adams. He definitely didn't follow the guidelines for methodological forecasting, such as assigning clear numerical predictions and tracking a Brier (or any chosen) scoring method.
As a result I see two main groups of thought on Adams: The first is forecasting oracle. The second is total charlatan (as far as I can tell this is the Rationalist viewpoint, I know SSC took this view).
I think the rationalist viewpoint is close to right. If we include the set of all semi-famous people who did/could speculate on an election (including Adams), and then imagine (we don't have the data) that we tracked all their predictions, with the knowledge that after the fact we would forget everyone who was wrong, Adams doesn't seem significantly correct.
But if Adams (or an abstracted idea of Adams argument) were correct, It would be because unlike current polling methods it allows for really high-dimensional data to be embedded into the forecast. As of now humans seem to be much better at getting a 'feel' for a movement than computers, because it requires using vast unrelated and unstructured data, which we specifically evolved to do* (I know we don't have great experiments to determine what we did/didn't specifically evolve for, so ignore this point if you want).
So, to that extent, current purely model-based election forecasts are at risk of having a severe form of omitted variable bias.
As an example, while polls are a little stable, Marine Le Pen is currently at a huge disadvantage: "According to a BVA poll carried out between Oct. 14 and Oct. 19, Le Pen would win between 25 percent and 29 percent of the vote in next April’s first round. If she faces Bordeaux mayor Alain Juppe -- the favorite to win the Republicans primary -- she’d lose the May 7 run-off by more than 30 percentage points. If it’s former President Nicolas Sarkozy, the margin would be 12 points."*
And yet PredictIt.org has her at ~40%. There is strong prior information from Brexit/Trump that seems important, but is absent in polls. It's almost as if we are predicting how people will change their mind when exposed to a 'treatment effect' of rightwing nationalism.
*http://www.bloomberg.com/news/articles/2016-11-16/french-pollsters-spooked-by-trump-but-still-don-t-see-le-pen-win
So then to tie this back to the original post, if you have stronger prior information, such as a strong reason to believe races will be 50-50, non-uniform priors, or that omitted variable bias exists, it would make sense to impose a structure on time-variation of the poll. I think this set of reasons is why it feels wrong to us when we see predictions varying so much far off from an election.
It's just masturbation with math notation.
That doesn't look like a reasonable starting point to me.
Going back to the OP...
Sure, but it's very difficult to model.
No, it's not. In a two-party system each party adjusts until it can capture close to 50% of the votes. There is a feedba... (read more)