You're looking at Less Wrong's discussion board. This includes all posts, including those that haven't been promoted to the front page yet. For more information, see About Less Wrong.

eli_sennesh comments on Open thread, Feb. 16 - Feb. 22, 2015 - Less Wrong Discussion

3 Post author: MrMind 16 February 2015 07:56AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (125)

You are viewing a single comment's thread.

Comment author: [deleted] 21 February 2015 12:33:22PM 0 points [-]

I want to look at deep neural-net learning and hierarchical inference through some kind of information-theoretic lens and try to show why hierarchical learning is such a powerful general principle. Anyone have an idea whether mutual information or KL-divergence is the normal measure used for this kind of study, or where I might look for literature other than surveys of deep learning, or why I might use one rather than the other?