Daniel_Burfoot comments on Mathematical simplicity bias and exponential functions - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (82)
Careful. It is reasonable to add complexity if the complexity is justified by increased explanatory power on a sufficiently large quantity of data. If you attempt to use a complex model to explain a small amount of data, you will end up overfitting the data. Note that this leaves us in a somewhat unpleasant situation: if there is a complex phenomenon regarding which we can obtain only small amounts of data, we may be forced to accept that the phenomenon simply cannot be understood.
Yes, this is exactly the point I was getting at when I wrote: "Of course it is best to test the model on new data."