and I think other mathematicians I've met are generally bad with numbers

Let me add another data point to your analysis: I'm a mathematician, and a visual thinker. I'm not particularly "good with numbers", in the sense that if someone says "1000 km" I have to translate that to "the size of France" before I can continue the conversation. Similarly with other units. So I think this technique might work well for me.

I do know my times tables though.