= 27d567bc2d48d9f40e9ccc20ea370b15)
SforSingularity comments on Mathematical simplicity bias and exponential functions - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (82)
There is a formal theory describing how to balance model complexity against fit to data: describe the model using a program on a simple, fixed turing machine, and then penalize that model by assigning a prior probability to it of 2^-L, where L is the length of the program...
this has all been worked out.
Which Turing machine, though?
How about picking one of these?
unpaywalled version
It can't be bettered on average, assuming that the thing you are modelling is computable.
But I haven't seen any proof to say that any other strategy will do worse on average. Anyone got any links?
See Hutter, Legg.
If I understand the maths right the important part of http://www.hutter1.net/ai/paixi.ps for using kolmogorov complexity is the part of section 2 that says
"The SPΘμ system is best in the sense that EnΘμ ≤ Enρ for any ρ."
That doesn't guarantee that this EnΘμ = Enρ for large numbers of different ρ. isn't true which would invalidate any claims of it being the one right way of doing things.
I was interested in links to papers with that theorem disproved.