I think the simplicity on this side of complexity is naive theories that "just make sense" and the simplicity on the other side of complexity is mathematical elegance.

When one of the commenters in the Amanda Knox thread said yesterday that the probability has to be either 0 or 1 because either she did it or she didn't, that sounds simple. The mathematics of Bayesian probability are also simple, in that they can be derived from a few premises and explain a wide variety of disparate situations. But they're not the same sort of simplicity.

Substitute understanding for complexity, maybe? I got the sense he was saying that simplifying before having a full understanding isn't valuable; but simplifying after a full understanding, having a simple model that still accurately describes the world, is extremely valuable.

The way I first took that quote:

Simple version: "Grass is always greener on the other side".

Complex version: Simplicity (aka Pattern, aka Information) is awesome, but becomes quickly boring and meaningless because it is KNOWN. It is the Simplicity/Pattern/Information which is currently hiding in the Chaos/Randomness which we're so eager for. It will, for a short while, be meaningful and interesting. Until we get used to it, too. Rinse and repeat.

On This Side would be that planets orbit circularly. The Complexity would be large numbers of circles within circles. On The Other Side would be that planets orbit elliptically.