cousin_it quoting Shalizi:
It's not the simplicity of the rules which matters, but the fact that simple rules are scarce in the space of all possible rules. If confines oneself to elaborate, baroque rules, but sticks to a particular style of baroque elaboration, one obtains the same effect.
But a particular style of baroque elaboration is one that has a short description.
Not necessarily. (Or did I misread your comment?) The particular style can have an arbitrarily long/complex description, and learning will still work as long as the class of described rules is small enough. This observation seems to imply that algorithmic simplicity doesn't play the central role I'd imagined it to play. This is precisely the point where I'd like to hear LW's informed replies.
I declare this Open Thread open for discussion of Less Wrong topics that have not appeared in recent posts.