I think the first three chapters of Jaynes are really excellent, and the chapters on updating (4 and 6 I think?), symmetry properties, and maximum entropy methods are also highly useful.
You will need some calculus (On the advanced end, you might need to understand how to use Lagrange multipliers, but it's not strictly necessary), but I think that's it. If you are unfamiliar with other concepts like Shannon entropy, what it might require is more practice of the material.
If you want to actually use this for data analysis, another modern textbook would be helpful (don't be afraid to read multiple books on the same thing).
Not too long ago, I asked LessWrong which math topics to learn. Eventually, I want to ask for what the prerequisites for each of those topics are and how I should go about learning them. This is a special case of that.
I'm rereading the sequences and Eliezer seems to love E.T. Jaynes. As part of my rationality self-study, I want to work my way through his Probability Theory: the Logic of Science. What math topics do I already need to understand to prepare myself for this? I learned calculus once upon a time, but not fantastically well, and I plan to start by reviewing that.
Also,
Despite Eliezer's praise of the "thousand-year-old vampire", it there a better book to learn probability theory?
Does anyone want to learn this (or the other math from my post above) with me? I'd love to have a partner or maybe even a work group. Location is no obstacle. [Two caveats: 1. I'm busy with stuff and may not be able to get into this for a few months 2. I hard, but I am incredibly slow at computation (such that on every math test I have ever taken, it took me at least 3 times as long as the second slowest person in the class to finish). You might find that I go to slow for you.]