Johnicholas comments on Mathematical simplicity bias and exponential functions - Less Wrong

12 Post author: taw 26 August 2009 06:34PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (82)

You are viewing a single comment's thread. Show more comments above.

Comment author: byrnema 27 August 2009 04:35:11PM *  6 points [-]

I have this vague idea that sometime in our past, people thought that knowledge was like an almanac; a repository of zillions of tiny true facts that summed up to being able to predict stuff about stuff, but without a general understanding of how things work. There was no general understanding because any heuristic that would begin to explain how things work would immediately be discounted by the single tiny fact, easily found, that contradicted it. Details and concern with minutia and complexity is actually anti-science for this reason. It’s not that details and complexity aren’t important, but you make no progress if you consider them from the beginning.

And then I wondered: is this knee-jerk reaction to dismiss any challenge of the keep-it-simple conventional wisdom the reason why we’re not making more progress in complex fields like biology?

For classical physics it has been the case that the simpler the hypothetical model you verify, the more you cash out in terms of understanding physics. The simpler the hypothesis you test, the easier it is to establish if the hypothesis is true and the more you learn about physics if it is true. However, what considering and verifying simpler and simpler hypotheses actually does is transfer the difficulty of understanding the real-world problem to the experimental set-up. To verify your super-simple hypothesis, you need to eliminate confounding factors from the experiment. Success in classical physics has occurred because when experiments were done, confounding factors could be eliminated through a well-designed set-up or were small enough to neglect. (Consider Galileo’s purported experiment of dropping two objects from a height – in real life that particular experiment doesn’t work because the lighter object may fall more slowly.)

In complex fields this type of modeling via simplification doesn’t seem to cash out as well, because it’s more difficult to control the experimental set-up and the confounding effects aren't negligible. So while I've always believed that models need to be simple, I would consider a different paradigm if it could work. How could understanding the world work any other way than through simple models?

Some method trends in biology: high through-put, random searches, brute force, etc.

Comment author: Johnicholas 27 August 2009 05:19:53PM 2 points [-]

Rather than describing the difference between physics and biology as "simple models" vs. "complex models", describe them in terms of expected information content.

Physicists generally expect an eventual Grand Unified Theory to be small in information content (one or a few pages of very dense differential equations, maybe as small as this: http://www.cs.ru.nl/~freek/sm/sm4.gif ). On the order of kilobytes, plus maybe some free parameters.

Biologists generally expect an eventual understanding of a species to be much much bigger. At the very least, the compressed human genome alone is almost a gigabyte; a theory describing how it works would be (conservatively) of the same order of magnitude.

All things being equal, would biologists prefer a yottabyte-sized theory to a zettabyte-sized theory? No, absolutely not! The scientific preference is still MOSTLY in the direction of simplicity.

There's a lot of sizes out there, and the fact that gigabyte-sized theories seem likely to defeat kilobyte-sized theories in the biological domain shouldn't be construed as a violation of the general "prefer simplicity" rule.

Comment author: timtyler 23 August 2010 05:51:24PM 0 points [-]

The uncompressed human genome is about 750 megabytes.

Comment author: Johnicholas 23 August 2010 10:17:40PM 0 points [-]

Thanks, and I apologize for the error.