eli_sennesh comments on Open thread, Feb. 16 - Feb. 22, 2015 - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (125)
I want to look at deep neural-net learning and hierarchical inference through some kind of information-theoretic lens and try to show why hierarchical learning is such a powerful general principle. Anyone have an idea whether mutual information or KL-divergence is the normal measure used for this kind of study, or where I might look for literature other than surveys of deep learning, or why I might use one rather than the other?