"Is it not true that when one rolls a fair 1d6, there is an actual 1/6 probability of getting any one specific value?"

No. The unpredictability of a die roll or coin flip is not due to any inherent physical property of the objects; it is simply due to lack of information. Even with quantum uncertainty, you could predict the result of a coin flip or die roll with high accuracy if you had precise enough measurements of the initial conditions.

Let's look at the simpler case of the coin flip. As Jaynes explains it, consider the phase space for the coin's motion at the moment it leaves your fingers. Some points in that phase space will result in the coin landing heads up; color these points black. Other points in the phase space will result in the coin landing tails up; color these points white. If you examined the phase space under a microscope (metaphorically speaking) you would see an intricate pattern of black and white, with even a small movement in the phase space crossing many boundaries between a black region and a white region.

If you knew the initial conditions precisely enough, you would know whether the coin was in a white or black region of phase space, and you would then have a probability of either 1 or 0 for it coming up heads.

It's more typical that we don't have such precise measurements, and so we can only pin down the coin's location in phase space to a region that contains many, many black subregions and many, many white subregions... effectively it's just gray, and the shade of gray is your probability for heads given your measurement of the initial conditions.

So you see that the answer to "what is the probability of the coin landing heads up" depends on what information you have available.

Of course, in practice you typically don't even have the lesser level of information assumed above -- you don't know enough about the coin, even in principle, to compute which points in phase space are black and which are white, or what proportion of the points are black versus white in the region corresponding to what you know about the initial conditions. Here's where symmetry arguments then give you P(heads) = 1/2.