If someone says that square circles exist and they have the math to prove it, do I need to check their math?

If someone had a theory that made useful predictions about the behaviour of reality, and could be used to make cool technology like transistors, and the only way you could get it to work and give those predictions was to assume the existence of hypothetical, mathematical square circles, who are you to call that theory "wrong" or "false"? The universe isn't obligated to be easy for us to understand, any more than it's obligated to be easy to understand for a mouse.