Here also is Einstein:

It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.

Or in the pithier paraphrase usually quoted:

Everything should be made as simple as possible, but no simpler.