= 27d567bc2d48d9f40e9ccc20ea370b15)
Gunnar_Zarncke comments on A Fervent Defense of Frequentist Statistics - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (125)
What he doesn't tell you is that myth 5 (online learning) applies McDiarmid's_inequality to derive the more precise bound
R is the real regret, R_emp the empirical regret (measured) and delta is the variance above.
This is derived e.g. in Introduction to Statistical Learning Theory, 2004, Bousquet et al
And you can get even better convergence if you know more about the distribution e.g. its VC dimension (roughly how difficult it is to decompose).
EDIT: LaTeX math in comment fixed.
Thanks for adding the reference. Are you sure McDiarmid's inequality applies in the online learning case, though? The inequality you wrote down looks like a uniform convergence result, which as far as I'm aware still required an i.i.d. assumption somewhere (although uniform convergence results are also super-awesome; I was even considering including them in my post but removed them due to length reasons).
Yes the formula is for the i.i.d case. See section 3.4 in the ref.