komponisto:

Numbers get people to convey more information about their beliefs. It doesn't matter whether you actually use numbers, or do something similar (and equivalent) like systematize the use of vague expressions. I'd be just as happy if people used a "five-star" system, or even in many cases if they just compared the belief in question to other beliefs used as reference-points.

If I understand correctly, you're saying that talking about numbers rather than the usual verbal expressions of certainty prompts people to be more careful and re-examine their reasoning more strictly. This may be true sometimes, but on the other hand, numbers also tend to give a false feeling of accuracy and rigor where there is none. One of the usual symptoms (and, in turn, catalysts) of pseudoscience is the use of numbers with spurious precision and without rigorous justification.

In any case, you seem to concede that these numbers ultimately don't convey any more information than various vague verbal expressions of confidence. If you want to make the latter more systematic and clear, I have no problem with that, but I see no way to turn them into actual numbers without introducing spurious precision.

The probability calculation you present should represent your brain's reasoning, as revealed by introspection. This is not a perfect process, and may be subject to later refinement. But it is definitely meaningful.

Trouble is, this is often not possible. Most of what happens in your brain is not amenable to introspection, and you cannot devise a probability calculation that will capture all the important things that happen there. Take your own example:

For example, consider my current probability estimate of 10^(-3) that Amanda Knox killed her roommate. On my current analysis, this is obtained as follows: I start with a prior of 10^(-4) (from a general homicide rate of about 10^(-3), plus reasoning that Knox is demographically an order of magnitude less likely to kill than the typical person; the figure happens to match my intuitive sense that I'd have to meet about 10,000 similar people before I'd have any fear for my life). Then all the evidence in the case raises the probability by about an order of magnitude at most, yielding 10^(-3).

See, this is where, in my opinion, you're introducing spurious numerical claims that are at best unnecessary and at worst outright misleading.

First you note that murderers are extremely rare, and that AK is a sort of person especially unlikely to be one. OK, say you can justify these numbers by looking at crime statistics. Then you perform a complex common-sense evaluation of the evidence, and your brain tells you that on the whole it's weak, so it's highly unlikely that AK killed the victim. So far, so good. But then you insist on turning this feeling of near-certainty about AK's innocence into a number, and you end up making a quantitative claim that has no justification at all. You say:

Now, this is just a rough order-of-magnitude argument. But it's already much more meaningful and useful than my just saying "I don't think she did it".

I strongly disagree. Neither is this number you came up with any more meaningful than the simple plain statement "I think it's highly unlikely she did it," nor does it offer any additional practical benefit. On the contrary, it suggests that you can actually make a mathematically rigorous case that the number is within some well-defined limits. (Which you do disclaim, but which is easy to forget.)

Even worse, your claims suggest that while your numerical estimates may be off by an order of magnitude or so, the model you're applying to the problem captures reality well enough that it's only necessary to plug in accurate probability estimates. But how do you know that the model is correct in the first place? Your numbers are ultimately based on an entirely non-mathematical application of common sense in constructing this model -- and the uncertainty introduced there is altogether impossible for you to quantify meaningfully.