then you want to use the simpler one - not because it's more likely to be 'true', but because it allows you to think more clearly.

Congratulations, you have now officially broken with Bayesianism and become a heretic. Your degree of belief in (prior probability of) a hypothesis should not depend on how clearly it allows you to think. Surely you can imagine all manner of ugly scenarios if that were the case.

Preferring to use a simpler theory doesn't require believing it to be more probable than it is. Expected utility maximization to the rescue.