User:potato smartly linked to Wei Dai's post "Frequentist Magic vs. Bayesian Magic", which along with its commentary would perhaps be the next thing to read after this post. (Wei Dai and user:potato think that (algorithmic) Bayesian magic is stronger but I agree with Toby Ord's and Vladimir Slepnev's points and would caution against hasty conclusions.)

More interestingly, probability theory models rational distributions of subjective degree of believe, and the optimal updating of degree of believe given new information. This is somewhat harder to show; dutch-book arguments do nicely to at least provide some intuitive understanding of the relation between degree of belief, betting, and probability, but there is still work to be done here.

In a sense there has been highly important and insightful work done in the vein of the Dutch book arguments, looking into e.g. how decisions should be made under what is called "indexical" or "anthropic" uncertainty, a situation where Bayesian reasoning seems like it ought to work but doesn't as such. This work has been done in large part by Less Wrong-cognizant folk. One easy-to-understand summary of some of the motivations for such research can be found found in User:ata's analysis of the classic "sleeping beauty" problem which is similar to User:potato's disambiguating approach used in the above post (and note Vladimir Nesov's similar warning against equivocation on the word "probability"). If you're interested in the foundations of justification (whether epistemic or moral, or what's the difference?), then you'd be wise to look into the big open questions in decision theory. There are a lot of 'em.