but in so far as you have it I'm not sure you can properly be said to have a prior at all.

I have an uncertain prior. I find that notion intuitive, it seems that you don't.

Your "uncertainty about that 0.6 number" is a meaningful notion only when there's something in (your model of) the world for it to be about.

It is uncertainty about the probability that the theory A is correct. I find the idea of "uncertainty about the probability" meaningful and useful.

I think that in a large number of cases you just do not have enough data for "figuring out all the intricate evidential structures" and the "heuristic technique" is all you can do. As for being rigorous, I'll be happy if in the limit it converges to the right values.

that's represented by your probability distribution over how the world is

But I don't have one. I'm not Omega -- the world is too large for me to have a probability distribution over it. I'm building models all of which are wrong but some of which are useful (hat tip to George Box). Is it useful to me to have multilayered models which involve probabilities of probabilities.

I think we are basically talking about whether to collapse all the meta-levels into one (your and Anders_H's position) or not collapse them (my position).