This post is an amendment and sort of apology for my LW posts dealing with complexity (1, 2). In these posts I got the terms all wrong. Now I'll try to set it right. Many thanks to taw, JGWeissman and Daniel_Burfoot.
The Solomonoff prior works like this: every bit string is assigned a real number that's equal to the probability of a "random program" outputting that string and then halting, when the "probability" of choosing each program is assumed to be inverse-exponential in its length. (Alternatively, you talk about a "universal prefix Turing machine" that consumes fair coin flips on the input tape.)
In this setup, the words "event", "hypothesis", "probability", "complexity" etc. are all ambiguous. For example, the word "hypothesis" can mean a) an individual program, b) an equivalence class of programs that output the same bit string, or c) a statement about output bit strings, like "the third bit will be 0". The word "event" has exactly the same problem.
Now the trouble is, you can give reasonable-sounding definitions of "probability" and "Kolmogorov complexity" to objects of all three types: (a), (b), and (c). But it's not at all clear what real-world implications these values should have. Does it make sense to talk about prior probability for objects of type (a), given that we can never distinguish two (a)'s that are part of the same (b)? (This is the mistake in saying that MWI has a higher prior probability than collapse interpretations.) Is it useful to talk about K-complexity for objects of type (c), given that a very long statement of type (c) can still have probability close to 1? (This is the mistake in saying Islam must be improbable because the Qur'an is such a thick book.) For that matter, is it at all useful to talk about K-complexity for objects of type (b) which are after all just bit strings, or should we restrict ourselves to talking about their prior probabilities for some reason?
At the moment it seems uncontroversial that we can talk about K-complexity for objects of type (a) - let's call them "programs" - and talk about prior probability for objects of types (b) and (c) - let's call them "sequences of observations" and "statements about the world", respectively. Curiously, this means there's no object for which we have defined both "complexity" and "prior probability", which makes all arguments of the form "complexity is high => probability is low" automatically suspect.
There's another worrying point. Given that "programs" get glued into equivalence classes - "sequences of observations" - anyway, it seems the K-complexity of an individual program is a completely inconsequential variable. You're better off just talking about the length. And the other two kinds of objects don't seem to have useful notions of K-complexity (according to my esteemed commentators who I thanked above), so we can adopt the general rule that mentioning K-complexity in a discussion of physics is always a sign of confusion :-)
What do you say? Is this better? I'm honestly trying to get better!
I understand being wary of the question, but imagine a Bayesian rationalist learning physics. First he learns about Classical Electromagnetism using fields, and he assigns this theory a prior probability based on the length of a program implementing this, and updates to a posterior probability based on the experimental evidence available. He then learns about the formulation based on potentials, which he finds makes all the same predictions, but has a shorter program length, so he adds some number of bits to his log-odds for the equivalence class, and he thinks that it is meaningless to ask whether reality uses potentials or fields, because no experiment could distinguish between them. Then he learns about Quantum Mechanics, which indicates that his classical understanding of potentials is an approximation of how they really work, that works on large scale, and that on the small scale where the approximation does not hold, fields are not a useful concept.
Should he be surprised to learn, having thought that experiment could not distinguish the two formulations, that there is an observable underlying reality in which uses potentials and not fields, and it is only in the domains he has previously encountered that fields make sense?
And then he learns Quantum Field Theory, and the shit really hits the fan.