I don't have a reference because the procedure is not rigorous; I came up with it off the top of my head. The intuition is that each of the contestants would've estimated the linear density of the jelly-beans, which is the same on all axes, and then cubed it, so you invert that by taking the cube root to get their actual estimates. To make this rigorous, you'd also have to account for the fact that the jar isn't actually a cube, which I have not done. I'd start by reducing the volume calculation to a bounding box (a cuboid) and a constant multiplicative factor, and assuming that everyone knows the correct constant factor for a cylinder. The length being different between the three dimensions does make a difference. I suspect (but have not proven) that having the jar, say, twice as tall as its diameter, would cause my procedure to act as though the error distribution for the height was twice as large.

If anyone knows of a source that handles this class of problem rigorously, please do post it. If not, perhaps it'd make a good exercise for someone looking for topics to write papers on.