Psy-Kosh comments on Conditional Independence, and Naive Bayes - Less Wrong

30 Post author: Eliezer_Yudkowsky 01 March 2008 01:59AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (15)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Psy-Kosh 02 March 2008 09:04:26PM 1 point [-]

This is neat. Two things though:

First a minor mathematical notation quibble/request, specifically about minus signs. Could you move them as much to the outside of a term as possible? What I mean is, for instance, you've got the following: H(S) = Sum i: p(Si)*-log2(p(Si))

Personally, I think I'd find H(S) = -Sum i: p(Si)*log2(p(Si)) easier to parse. Probably just due to what I'm used to seeing in mathematical notation, but I seem to see the minus in the middle "before" I see the multiplication, so my initial reaction is "wait, what? subtracting those? huh?" before the rest of my brain catches up and sees it's actually the expected multiplication by a negated term, just that the negation has been placed in an unusual spot.

Maybe that's just me though.

Second, just wondering if there's a simple general principle for knowing when, given a bunch of mutually correlated variables, we can construct a central variable like the above that shields everything from everything else (either completely or at least to a good extent. (Obviously this is tied to how well stuff clusters in conceptspace, but I meant if there's any _relatively simple_ mathematical criteria)