The distinction the author draws is that the second box in the earlier problem really does have to be true or false, since "it is a historic statement about the physical world"

If one of the boxes says that exactly one of them was written by Alice, and you know from another source that Alice always tells the truth, Bob always lies, and both boxes were inscribed by one of them, and Alice and Bob never say anything self-referential, then this is correct.

If it says that one of the boxes was labelled by someone who always tells the truth, then it's not just talking about the person who labelled that box. It's also talking about every aspect of reality that they've ever referenced, and if they were the one to write that inscription, then it's self-referential.