Comment on Is simplicity truth indicative?

Thu, 13 Aug 2015 15:39:04 GMT

noise and simplicity are opposites (information entropy), thus if we dislike noise we should like simplicity. Does this seem like a reasonable perspective?

Not quite. Noise and simplicity are not opposites. I would say that the amount of noise in the data (along with the amount of data) imposes a limit, an upper bound, on the complexity that you can credibly detect.

Basically, if your data is noisy you are forced to consider only low-complexity models.

Discuss