I'd unify your "So What" with "meaningless" into a single category "does not constrain observations". Math passes the test inasmuch as it constrains observations about outcomes of proof checking.
But now some people will complain (are already complaining) that we reject the majority of humanity's thought.
Again, it does seem observable that nobody has explained why three is prime and four isn't. (I'm not sure you can actually use 'why' in an intelligible way here; possibly I'm being confused by non-mathematical language applied to math.) It's not an observation I would expect anyone to care about, and possibly it may be the equivalent of nobody having seen something invisible; but it does seem to make a statement that could in principle have gone the other way.
David Stove's "What Is Wrong With Our Thoughts" is a critique of philosophy that I can only call epic.
The astute reader will of course find themselves objecting to Stove's notion that we should be catologuing every possible way to do philosophy wrong. It's not like there's some originally pure mode of thought, being tainted by only a small library of poisons. It's just that there are exponentially more possible crazy thoughts than sane thoughts, c.f. entropy.
But Stove's list of 39 different classic crazinesses applied to the number three is absolute pure epic gold. (Scroll down about halfway through if you want to jump there directly.)
I especially like #8: "There is an integer between two and four, but it is not three, and its true name and nature are not to be revealed."