Despite other suggestions, I encourage you to read the whole book, not just the first chapters. It will give you a very solid understanding of theoretical probability theory, on things like "why the normal", group symmetry priors, nonconglomerability, A_p distributions, etc.
On the other hand, when you have read the first 6 chapters of Jaynes, go look at the first part of Sivia, you'll see Jaynes' concepts laid out with more simpler examples and a high degree of clarity.
The second part of Sivia and the Gelman/Carlin/Stern/Rubin could be used to gain a deeper understanding of what modern probabilistic models actually are and how they are used in practice.
Be aware though, after Jaynes, tolerating the sloppy thinking that is endemic in the field would be much harder.
Not too long ago, I asked LessWrong which math topics to learn. Eventually, I want to ask for what the prerequisites for each of those topics are and how I should go about learning them. This is a special case of that.
I'm rereading the sequences and Eliezer seems to love E.T. Jaynes. As part of my rationality self-study, I want to work my way through his Probability Theory: the Logic of Science. What math topics do I already need to understand to prepare myself for this? I learned calculus once upon a time, but not fantastically well, and I plan to start by reviewing that.
Also,
Despite Eliezer's praise of the "thousand-year-old vampire", it there a better book to learn probability theory?
Does anyone want to learn this (or the other math from my post above) with me? I'd love to have a partner or maybe even a work group. Location is no obstacle. [Two caveats: 1. I'm busy with stuff and may not be able to get into this for a few months 2. I hard, but I am incredibly slow at computation (such that on every math test I have ever taken, it took me at least 3 times as long as the second slowest person in the class to finish). You might find that I go to slow for you.]