You are not allowed to pick whatever language you want, you have to pick the optimal code. If in the most concise code possible, state 'a' has a smaller code than state 'b', then 'a' must be more probable than 'b', since the most concise codes possible assign the smallest codes to the most probable states.

I haven't read anything like this in my admittedly limited readings on Solomonoff induction. Disclaimer: I am only a mere mathematician in a different field, and have only read a few papers surrounding Solomonoff.

The claims I've seen revolve around "assembly language" (for some value of assembly language) being sufficiently simple that any biases inherent in the language are small (some people claim constant multiple on the basis that this is what happens when you introduce a symbol 'short-circuiting' a computation). I think a more correct version of Anti-reductionist's argument should run, "we currently do not know how the choice of language affects SI; it is conceivable that small changes in the base language imply fantastically different priors."

I don't know the answer to that, and I'd be very glad to know if someone has proved it. However, I think it's rather unlikely that someone has proved it, because 1) I expect it will be disproven (on the basis that model-theoretic properties tend to be fragile), and 2) given the current difficulties in explicitly calculating SI, finding an explicit, non-trivial counter-example would probably be difficult.

Note that

Choose a language that can describe MWI more easily than Copenhagen, and they say you should believe MWI; choose a language that can describe Copenhagen more easily than MWI, and they say you should believe Copenhagen.

is not such a counter-example, because we do not know if "sufficiently assembly-like" languages can be chosen which exhibit such a bias. I don't think the above thought-experiment is worth pursuing, because I don't think we even know a formal (on the level of assembly-like languages) description of either CI or MWI.