To whom it may concern:
This thread is for the discussion of Less Wrong topics that have not appeared in recent posts. If a discussion gets unwieldy, celebrate by turning it into a top-level post.
(After the critical success of part II, and the strong box office sales of part III in spite of mixed reviews, will part IV finally see the June Open Thread jump the shark?)
I agree about Jaynes and the exactness of Bayesian inference. (I haven't read his Probability Theory fully, but I should definitely get to it sometime. I did got through the opening chapters however, and it's indeed mighty convincing.) Yet, I honestly don't see how either Jaynes or your comments answer my question in full, though I seen no significant disagreement with what you've written. Let me try rephrasing my question once more.
In natural sciences, when you characterize some quantity with a number, this number must make sense in some empirical way, testable in an experiment, or at least with a thought experiment if a real one isn't feasible in practice. Suppose that you've determined somehow that the temperature of a bowl of water is 300K, and someone asks you what exactly this number means in practice -- why 300, and not 290, or 310, or 299, or 301? You can reply by describing (or even performing) various procedures with that bowl of water that will give predictably different outcomes depending on its exact temperature -- and the results of some such procedures with this particular bowl are consistent only with a temperature of 300K plus/minus some small value that can be made extremely tiny with a careful setup, and not any other numbers.
Note that the question posed here is not how we've determined what the temperature of the water is in the first place. Instead, the question is: once we've made the claim that the temperature is some particular number, what practical observation can we make that will show that this particular number is consistent with reality, and others aren't? If an number can't be justified that way, then it is not something science can work with, and there is no reason to consider one value as "correct" and another "incorrect."
So now, when I ask the same question about probability, I'm not asking about the procedures we use to derive these numbers. I'm asking: once we've made the claim that the probability of some event is p, what practical observations can we make that will show that this particular number is consistent with reality, and others aren't -- except by pointing to frequencies of events? I understand how we would reach a probability figure in the Elbonia vs. Ruritania scenario, I agree that Bayesian inference is an exact system, and I see what the possible sources of disagreement could be and how they should be straightened out when asymmetrical information is eliminated. I am not arguing with any of that (at least in the present context). Rather, what I'd like to know is whether the figures ultimately reached make any practical sense in terms of some observable properties of the universe, except for the frequentist ratios predicted by them. (And if the latter is the only answer, this presumably means that any sensible interpretation of probability would have to incorporate significant frequentist elements.)
That question, interesting as it is, is above my pay grade; I'm happy enough when I get the equations to line up the right way. I'll let others tackle it if so inclined.