= 27d567bc2d48d9f40e9ccc20ea370b15)
taiyo comments on Information theory and the symmetry of updating beliefs - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (28)
There is a sign problem when iev(A,B) is defined. You mention that you can get the mutual information of A and B by taking -log_2 of probabilistic evidence pev(A,B) = P(AB) / [P(A)P(B)], but this creates an extra negative sign:
-log2(pev(A,B)) = -log2[P(AB) / [P(A)P(B)]] = -[ log2(P(AB)) - [log2(P(A)) + log2(P(B))] ] = -log2(P(AB)) + log2(P(A)) + log2(P(B)) = inf(AB) - inf(A) - inf(B).