Upon thinking about that second definition of rational neutrality, I find myself thinking that that can't be right. It's identical to calibration. And even a rational-neutral agent that's been "repaired" by applying the best possible probability estimate adjustment function will still return the same ordinal probabilities: Barney the Biased, even after adjustment, will return higher probabilities for statements he is biased toward than statements he is biased against.

I would have said this:

So, here's another definition of rational neutrality I came up with by writing this: you are rational-neutral if, given only your source code and your probability estimates, it's impossible for someone to come up with better probability estimates.

...but that definition doesn't rule out the possibility that an agent would look at your probability estimates, figure out what the problem is, and come up with a better solution on its own. In the extreme case, no agent would be considered rational-neutral unless it had a full knowledge of all mathematical results. That's not what I want; therefore, I stick by my original definition.