This is different than most slippery philosophical problems -- the math actually fights back in a revealing way.

I don't see any particular asymmetry, actually. (Which is no surprise when you realize that I consider mathematics to be rigorous philosophy.) Sometimes the way it fights back is (sufficiently) revealing, and sometimes it isn't.

There remain deeply mysterious unsolved problems in mathematics, for which merely fiddling with existing tools has not produced answers. The point of view I take (which is implicitly advocated by this post) is that whenever you have a problem that you can't solve, it's because your existing tools are inadequate, and you need to develop better tools. How does one develop better tools? Well, you can hope to discover them by accident in the course of analyzing the unsolved problem, or you can try to develop them systematically by figuring out how to better solve problems you are already able to solve. The latter is my preferred approach.

(I would also recommend this comment for context.)