Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Venkat comments on Occam's Razor - Less Wrong

37 Post author: Eliezer_Yudkowsky 26 September 2007 06:36AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (52)

Sort By: Old

You are viewing a single comment's thread.

Comment author: Venkat 26 September 2007 09:33:02PM 3 points [-]

What you are talking about in terms of Solmonoff induction is usually called algorithmic information theory and the shortest-program-to-produce-a-bit-string is usually called Kolmogorov-Chaitin information. I am sure you know this. Which begs the question, why didn't you mention this? I agree, it is the neatest way to think about Occam's razor. I am not sure why some are raising PAC theory and VC-dimension. I don't quite see how they illuminate Occam. Minimalist inductive learning is hardly the simplest "explanation" in the Occam sense, and is actually closer to Shannon entropy in spirit, in being more of a raw measure. Gregory Chaitin's 'Meta Math: The Search for Omega', which I did a review summary of is a pretty neat look at this stuff.