It's interesting to think about entropy from this perspective because it becomes related to the subjectivist interpretation of probability.

If you haven't already read Jaynes derivation of maxent, and the further derivation of much of statistical mechanics from those principles, that would be a good place to start.