= 27d567bc2d48d9f40e9ccc20ea370b15)
Wei_Dai comments on Open Thread: October 2009 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (425)
This is a subtle point. The NFL theorem does prohibit any algorithm from doing well over all possible worlds. But Solomonoff induction does well on any world that has any kind of computable regularity. If there is no computable regularity, then no prior can do well. In fact, the Solomonoff prior does just as well asymptotically as any computable prior.
As is often the case, thinking in terms of codes can clear up the issue. A world is a big data file. Certainly, an Earth-specific algorithm can get good compression rates if it is fed data that comes from Earth. But as the data file gets large, the Solomonoff general-purpose compression algorithm will achieve compression rates that are nearly as good; in the worst case, just has to prepend the code of the Earth-specific algorithm to its encoded data stream, and it only underperforms by that program size.
The reason AIXI doesn't work in practice is that the "efficient approximations" aren't really efficient, or aren't good approximations.
This seems to be a common belief. But see this discussion I had with Eliezer where I offered some good arguments and counterexamples against it.
The link goes to the middle, most relevant part of the discussion. But if you look at the top of it, I'm not arguing against the Solomonoff approach, but instead trying to find a generalization of it that makes more sense.
I've linked to that discussion several times in my comments here, but I guess many people still haven't seen it. Maybe I should make a top-level post about it?