wobster109 comments on Conditional Independence, and Naive Bayes - Less Wrong

30 Post author: Eliezer_Yudkowsky 01 March 2008 01:59AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (15)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: wobster109 26 March 2011 05:27:31PM 1 point [-]

This was what I expected to see, and I believe it's equivalent to H(X,Y,Z) = H(X) + H(Y) + H(Z) - I(X;Z) - I(Z;Y) - I(X;Y | Z)

It appears that Z is very artificially constructed --- Z is exactly I(X,Y) in the example. Therefore, H(X,Y) = H(X,Y,Z). Since the term I(X,Y | Z) is mutual information about X and Y given Z, that's just 0. There's no new mutual information about X and Y that isn't already in Z. So I believe that we could replace it with +I(X,Y) - I(X,Y,Z), and get inclusion-exclusion.