= 27d567bc2d48d9f40e9ccc20ea370b15)
Vaniver comments on A History of Bayes' Theorem - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (85)
I think the premise is that, if you are weighting the importance of polls based on how well the polls predicted past elections, you are using the spirit of Bayes, and the only consistent and correct way to do it mathematically is some form of Bayes itself.
IIRC his weights were based on objective quality metrics like sample size and recency.
When you say "objective quality metrics," how can they be determined to be such without using prior knowledge?
For sample size, it's actually objectively measurable. For recency etc. you can just use your expert judgment and validate against data with ad hoc techniques.
Ask Nate Silver for details if you wish. He never indicated he has a big Bayesian model behind all that.
You reach a point very early where model uncertainty makes Bayesian methods no better than ad hoc methods.
I don't mean to argue that Nate Silver had a "big Bayesian model behind all that." But if sample size and recency increase the reliability of polls, you can objectively measure how much they do and it seems that using Bayesian methods you could create an objectively best prior weighting system, which seems like the point that Vaniver was making.
I'm not immediately familiar with the math but it seems odd to me that it would be much more work to do a regression for a "best prior" than to come up with an ad hoc method, especially considering that "expert judgment" tends to be really bad (at least according to Bishop and Trout ).
Of course, I should probably wait to disagree until he [Nate Silver] gets something wrong.