I'm not sure what your background is; there are a number of books in philosophy of statistics that might be more accessible than Jaynes. I'm with the others - you should study some of the simpler foundations of statistics before diving into Bayesianism.
1) Luce and Raiffa's Games and Decisions (1989) introduces von Neumann and Morgenstern's axioms of objective utility theory and game theory; the book is geared towards serious social scientists.
2) Savage's The Foundations of Statistics (1954) is much harder than Luce and Raiffa, but it introduces an axiomatic system of subjective utility theory that complements objective utility theory well.
3) Fagin, Halpern, Moses and Vardi's Reasoning About Knowledge (2004) contains a reasonable introduction to statistics via the Kolmogorov axiomatization. It is suitable for technically minded philosophy students who have no background in measure theory, set theory or analysis.
4) Modern statistics builds heavily on calculus. You may want to brush up - I really like Bressoud's A Radical Approach to Real Analysis (2006). If you find yourself into the Kolmogorov axiomatization, you should also check out Bressoud's A Radical Approach to Lebesgue Measure Theory (2008).
These books should give you some perspective on the different traditions in philosophy of statistics and rational decision theory. After this Jaynes should make a lot of sense - he belongs to the "subjective" camp that Savage pioneered. However, it is important to understand when studying philosophy of statistics that there are multiple camps and they have different perspectives.
Not too long ago, I asked LessWrong which math topics to learn. Eventually, I want to ask for what the prerequisites for each of those topics are and how I should go about learning them. This is a special case of that.
I'm rereading the sequences and Eliezer seems to love E.T. Jaynes. As part of my rationality self-study, I want to work my way through his Probability Theory: the Logic of Science. What math topics do I already need to understand to prepare myself for this? I learned calculus once upon a time, but not fantastically well, and I plan to start by reviewing that.
Also,
Despite Eliezer's praise of the "thousand-year-old vampire", it there a better book to learn probability theory?
Does anyone want to learn this (or the other math from my post above) with me? I'd love to have a partner or maybe even a work group. Location is no obstacle. [Two caveats: 1. I'm busy with stuff and may not be able to get into this for a few months 2. I hard, but I am incredibly slow at computation (such that on every math test I have ever taken, it took me at least 3 times as long as the second slowest person in the class to finish). You might find that I go to slow for you.]