= 27d567bc2d48d9f40e9ccc20ea370b15)
Daniel_Reeves comments on Conditional Independence, and Naive Bayes - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (15)
I'm confused :/
P(X,Y,Z) = P(X,(Y,Z)) = P(X) + P(Y,Z) - P(X;(Y,Z)) = P(X) + (P(Y) + P(Z) - P(Y;Z)) - P(X;(Y,Z)) = P(X) + (P(Y) + P(Z) - P(Y;Z)) - P((X;Y),(X;Z)) = P(X) + (P(Y) + P(Z) - P(Y;Z)) - (P(X;Y) + P(X;Z) - P(X;Y;Z)) = P(X) + P(Y) + P(Z) - P(X;Y) - P(Y;Z) - P(X;Z) + P(X;Y;Z)
By the inclusion-exclusion principle, no?
This was what I expected to see, and I believe it's equivalent to H(X,Y,Z) = H(X) + H(Y) + H(Z) - I(X;Z) - I(Z;Y) - I(X;Y | Z)
It appears that Z is very artificially constructed --- Z is exactly I(X,Y) in the example. Therefore, H(X,Y) = H(X,Y,Z). Since the term I(X,Y | Z) is mutual information about X and Y given Z, that's just 0. There's no new mutual information about X and Y that isn't already in Z. So I believe that we could replace it with +I(X,Y) - I(X,Y,Z), and get inclusion-exclusion.