= 27d567bc2d48d9f40e9ccc20ea370b15)
timtyler comments on Open Thread: December 2009 - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (263)
Is there a proof anywhere that occam's razor is correct? More specifically, that occam priors are the correct priors. Going from the conjunction rule to P(A) >= P(B & C) when A and B&C are equally favored by the evidence seems simple enough (and A, B, and C are atomic propositions), but I don't (immediately) see how to get from here to an actual number that you can plug into Baye's rule. Is this just something that is buried in textbook on information theory?
On that note, assuming someone had a strong background in statistics (phd level) and little to no background in computer science outside of a stat computing course or two, how much computer science/other fields would they have to learn to be able to learn information theory?
Thanks to anyone who bites
Occam's razor is dependent on a descriptive language / complexity metric (so there are multiple flavours of the razor).
Unless a complexity metric is specified, the first question seems rather vague.
I think you might be making this sound easier than it is. If there are an infinite number of possible descriptive languages (or of ways of measuring complexity) aren't there an infinite number of "flavours of the razor"?
Yes, but not all languages are equal - and some are much better than others - so people use the "good" ones on applications which are sensitive to this issue.
There's a proof that any two (Turing-complete) metrics can only differ by at most a constant amount, which is the message length it takes to encode one metric in the other.
Of course, the constant can be arbitrarily large.
However, there are a number of domains for which this issue is no big deal.