You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

orthonormal comments on [Link] Better results by changing Bayes’ theorem - Less Wrong Discussion

3 Post author: XiXiDu 09 March 2012 07:38PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (17)

You are viewing a single comment's thread. Show more comments above.

Comment author: orthonormal 10 March 2012 04:01:33PM 2 points [-]

This could possibly be explained if the heuristic also embodied some background information that allows us to correct for overconfidence if P(f|e) that isn't explicitly mentioned in the data

Exactly. If there's some structure to the full dataset that's unrecoverable from the part that's kept, you can code that structure into a heuristic which will outperform Naive Bayes on the remaining data- but an ideal Bayesian reasoner with access to the full dataset would have picked up that structure as well, and you wouldn't be outperforming xer.

So the post is evidence of interesting structure in word frequencies, not a deficiency of Bayes' Rule.