= 27d567bc2d48d9f40e9ccc20ea370b15)
MichaelBishop comments on Mathematical simplicity bias and exponential functions - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (82)
To me, the problem is not "Mathematical Simplicity Bias," but rather, failing to check the model with empirical data. It seems totally reasonable to start with a simple model and add complexity necessary to explain the phenomenon. (Of course it is best to test the model on new data.)
Also, if you're going to claim Mathematical Simplicity Bias is, "One of biases that are extremely prevalent in science," it would help to provide real examples of scientists failing because of it.
Careful. It is reasonable to add complexity if the complexity is justified by increased explanatory power on a sufficiently large quantity of data. If you attempt to use a complex model to explain a small amount of data, you will end up overfitting the data. Note that this leaves us in a somewhat unpleasant situation: if there is a complex phenomenon regarding which we can obtain only small amounts of data, we may be forced to accept that the phenomenon simply cannot be understood.
Yes, this is exactly the point I was getting at when I wrote: "Of course it is best to test the model on new data."