(continued..)

Explanations?

Alexander analyzes the difference between $p_1$ and $p_2$ in terms of the famous "explaining away" effect. Alexander supposes that $p_2$ has learned some "causes":

The reason is that the probability mass provided by examples x₁, ..., xₙ such that ϕ(xᵢ) holds is now distributed among the universal statement ∀x:ϕ(x) and additional causes Cⱼ known to the more powerful agent that also imply ϕ(xᵢ). Consequently, ∀x:ϕ(x) becomes less "necessary" and has less relative explanatory power for the more informed agent.

An implication of this perspective is that if the weaker agent learns about the additional causes Cⱼ, it should also lower its credence in ∀x:ϕ(x).

Postulating these causes adds something to the scenario. One possible view is that Alexander is correct so far as Alexander's argument goes, but incorrect if there are no such $C_j$ to consider.

However, I do not find myself endorsing Alexander's argument even that far.

If $C_1$ and $C_2$ have a common form, or are correlated in some way -- so there is an explanation which tells us why the first two sentences, $\varphi \left(x_1\right)$ and $\varphi \left(x_2\right)$, are true, and which does not apply to $n>2$ -- then I agree with Alexander's argument.

If $C_1$ and $C_2$ are uncorrelated, then it starts to look like a coincidence. If I find a similarly uncorrelated $C_3$ for $\varphi \left(x_3\right)$, $C_4$ for $\varphi \left(x_4\right)$, and a few more, then it will feel positively unexplained. Although each explanation is individually satisfying, nowhere do I have an explanation of why all of them are turning up true.

I think the probability of the universal sentence should go up at this point.

So, what about my "conditional probabilities also change" variant of Alexander's argument? We might intuitively think that $\varphi \left(x_1\right)$ and $\varphi \left(x_2\right)$ should be evidence for the universal generalization, but $p_2$ does not believe this -- its conditional probabilities indicate otherwise. 

I find this ultimately unconvincing because the point of Paul's example, in my view, is that more accurate priors do not imply more accurate posteriors. I still want to understand what conditions can lead to this (including whether it is true for all notions of "accuracy" satisfying some reasonable assumptions EG proper scoring rules).

Another reason I find it unconvincing is because even if we accepted this answer for the paradox of ignorance, I think it is not at all convincing for the problem of old evidence. 

What is the 'problem' in the problem of old evidence?

... to be further expanded later ...

The matter seems terribly complex and interesting to me.

Notions of Accuracy?

Suppose $p_1$ is a prior which has uncertainty about $\varphi \left(x_1\right),\varphi \left(x_2\right),...$ and uncertainty about ${\forall }_n\varphi \left(x_n\right)$. This is the more ignorant prior. Consider $p_2$ some prior which has the same beliefs about the universal statement -- $p_1({\forall }_n\varphi \left(x_n\right))=p_2({\forall }_n\varphi \left(x_n\right))$ -- but which knows $\varphi \left(x_1\right)$ and $\varphi \left(x_2\right)$.

We observe that $p_1$ can increase its credence in the universal statement by observing the first two instances, $\varphi \left(x_1\right)$ and $\varphi \left(x_2\right)$, while $p_2$ cannot do this -- $p_2$ needs to wait for further evidence. This is interpreted as a defect.

The moral is apparently that a less ignorant prior can be worse than a more ignorant one; more specifically, it can learn more slowly.

However, I think we need to be careful about the informal notion of "more ignorant" at play here. We can formalize this by imagining a numerical measure of the accuracy of a prior. We might want it to be the case that more accurate priors are always better to start with. Put more precisely: a more accurate prior should also imply a more accurate posterior after updating. Paul's example challenges this notion, but he does not prove that no plausible notion of accuracy will have this property; he only relies on an informal notion of ignorance.

So I think the question is open: when can a notion of accuracy fail to follow the rule "more accurate priors yield more accurate posteriors"? EG, can a proper scoring rule fail to meet this criterion? This question might be pretty easy to investigate.

Conditional probabilities also change?

I think the example rests on an intuitive notion that we can construct $p_2$ by imagining $p_1$ but modifying it to know $\varphi \left(x_1\right)$ and $\varphi \left(x_2\right)$. However, the most obvious way to modify it so is by updating on those sentences. This fails to meet the conditions of the example, however; $p_2$ would already have an increased probability for the universal statement.

So, in order to move the probability of $\varphi \left(x_1\right)$ and $\varphi \left(x_2\right)$ upwards to 1 without also increasing the probability of the universal, we must do some damage to the probabilistic relationship between the instances and the universal. The prior $p_2$ doesn't just know  $\varphi \left(x_1\right)$ and $\varphi \left(x_2\right)$; it also believes the conditional probability of the universal statement given those two sentences to be lower than $p_1$ believes them to be.

It doesn't think it should learn from them!

This supports Alexander's argument that there is no paradox, I think. However, I am not ultimately convinced. Perhaps I will find more time to write about the matter later.

Noting that images currently look broken to me, in this post.

I don't really interact with Twitter these days, but maybe you could translate my complaints there and let me know if you get any solid gold?

I've tried writing the beginning of a paper that I want to read the rest of, but the LLM did not complete it well enough to be interesting.