timtyler comments on How to Be Oversurprised - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (14)
What's optimal in terms of keeping information about the world is to store your entire sense data history - and everything you were born with. That is often extremely wasteful of memory (even if you compress) - but surely much more compact than keeping track of all the possible hypotheses that might explain your sense data, and their respective probabilities. I think it is hard to come up with a sense in which that is optimal.
You know, non-parametric methods along the lines of k-nearest neighbors basically do this. The cleverness in these methods is finding a log(n) access scheme for your data (a database problem, essentially, rather than a machine learning problem).