wobster109 comments on Conditional Independence, and Naive Bayes - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (15)
This was what I expected to see, and I believe it's equivalent to H(X,Y,Z) = H(X) + H(Y) + H(Z) - I(X;Z) - I(Z;Y) - I(X;Y | Z)
It appears that Z is very artificially constructed --- Z is exactly I(X,Y) in the example. Therefore, H(X,Y) = H(X,Y,Z). Since the term I(X,Y | Z) is mutual information about X and Y given Z, that's just 0. There's no new mutual information about X and Y that isn't already in Z. So I believe that we could replace it with +I(X,Y) - I(X,Y,Z), and get inclusion-exclusion.