That simplicity in the information sense equates with parsimony is most unlikely; for one thing, simplicity is dependent on choice of language--an insight that should be almost intuitive to a rationalist.

You didn't read that sequence at all, and probably don't actually know what simplicity means in an information-theoretic sense.

To be fair, that sequence doesn't really answer questions about choice-of-language; it took reading some of Solomonoff's papers for me to figure out what the solution to that problem is.