It has a germ of truth, but I think it's deeply misleading. In particular, it needs some kind of nod to the importance of relevance to everyday life. E.g., it would be more serious to claim "all science is either physics, or the systematizing side of some useful discipline like engineering, or stamp collecting." Pure stamp collecting endeavors have nothing to stop them from veering into the behavior stereotypically associated with modern art or the Sokal hoax. Fields like paleobotany or astronomy (or, indeed, physics itself in near-unobservable limits) can become arbitrarily pure stamp collecting when the in-group controls funding. More applied fields like genetics or immunology or synthetic chemistry or geology are messy and disordered compared to pure physics, and do resemble stamp collecting in that messiness. But true stamp collecting is not merely messy, but also arbitrarily driven by fashion. To the extent that a significant amount of the interest (and money) associated with an academic field flows from applications like agriculture and medicine and resource extraction, it tends not to dive so deeply into true free-floating arbitrariness of pure stamp collecting.