No, I thought the physical referent for the integer math was something like "Human mental instantiation of an idea that is reasonably agreed upon." I believe you are referring to the physical referent of the preimage of the isomorphism (i.e., the physical system itself. A somewhat redundant thing to call a referent, since it is actually the explicit meaning of the statement.)