What are the reasons that mathematicians like to appeal to a mathematical realm?

In my case, I feel like I 'manipulate' mathematical objects in my mind as one would manipulate physical objects. Also, I feel like I 'explore' a mathematical subject as one would explore a territory ... I investigate how it works rather than make up how it works. (If any one else feels like belief in a math realm is natural, even if illusory, what are your reasons?)

The temptation to appeal to an immaterial realm may also relate to your 4th point:

4) It has always been the case that if you set up some physical system isomorphic to some mathematical operation, performed the corresponding physical operation, and re-interpreted it by the same isomorphism, the interpretation would match that which the rules of math give (though again counterfactual, as there's no one to be observing or setting up such a system).

It has always been the case. But why? Why do mathematical isomorphisms have to follow the same rules as their physical counterparts?

My guess, completely unsubstantiated by any knowledge of neuroscience, is that when mathematicians do math, they are creating and manipulating physical models in their brains. We explore the logic embedded in physical reality by studying physics on a smaller scale, in a much more abstract way, in our brains. Because -- and this is my only argument -- how else would we know? I would guess that the models are implemented at the cellular (neuronal) scale rather than sub-cellular.

So then there would indeed good reasons for our sense of a Platonic realm. The Platonic realm would be the special software (hardware?) that we run when we think about and develop mathematics.