(I'm annoyed that I can't find a reference to double check this now.)

I just got lucky and found a reference in Harriet Zuckerman's Scientific Elite: Nobel Laureates in the United States:

• Robert Marc Friedman (1981). Nobel Physics Prize in perspective. Nature, 292, 793-798.

It's hard to make a comprehensive selection of relevant extracts, but here are some bits & pieces, since the article's paywalled:

I shall argue that the subsequent restriction of the scope of the physics prize stemmed from a determined effort within the Swedish physics discipline. Not surprisingly, Nobel Prize deliberations became enmeshed in the process by which various factions within the Swedish physical science community attempted to define and to legitimize their particular notions of physics: its objects, methods and goals.

[During the debate over awarding Einstein the Physics Prize], three of the committee's five members belonged to the Uppsala tradition of experimental physics — B. Hasselberg, G. Granqvist, and A. Gullstrand. The others were S. Arrhenius (physical chemistry) and V. Carlheim-Gyllensköld (mathematical and cosmical physics). The committee's strong experimentalist bias proved significant from the start in determining Nobel Prize decisions and for interpreting the statutes.

Debates on the role of mathematics and theory in physics that had begun in the 1890s carried over into deliberations on the prize. Hence, even in the case of a world-wide campaign for Henri Poincaré, in which his mathematical physics qua physics was delineated from his purely mathematical accomplishments, the committee did not see fit to recommend him for the prize⁸. In a protest note, Carlheim-Gyllensköld pointed out the committee's general unwillingness and inability during the first ten years to evaluate nominations in mathematical physics such as those of Boltzmann, Heaviside, Kelvin and Poynting⁹. Arrhenius had commented that Uppsala physicists consider "spectral analysis...the only part of physics worth pursuing"¹⁰. Indeed, Hasselberg did "all in my power to procure the prize" (1907) for A. A. Michelson, who had received but two nominations, not for his role in the aether-drift experiment but, rather, primarily for his work in precision spectroscopy and metrology¹¹.

Other examples abound. Thus the proposal for the 1908 prize was sabotaged partly by those who believed it "unjust" to award a prize to a theoretician, M. Planck, without dividing the honour with an experimentalist.¹³

The Einstein deliberation came at a turning point in the development of Swedish physics and of the committee. By 1923, after the deaths of Hasselberg and Granqvist, Oseen was elected to the committee as a regular member and was joined by his Uppsala colleague, atomic physicist Manne Siegbahn. Together with Oseen's friend, Gullstrand, the three Uppsala physicists commanded a majority on the committee [...] By promoting atomic physics, in which experiment and theory intimately progress together, he [Oseen] hoped to overcome traditional prejudices and to institutionalize a theoretical physics standing apart from abstract mathematical physics and mechanics. Yet, to be successful [...] they not only had to promote and legitimize their preferred research programmes, but would also have to eliminate what they considered insignificant specialties. Starting in the 1920s, the Uppsala group began a determined campaign to restrict the definition of physics within the Academy and with regard to the Nobel Prize.

in 1923, the new Uppsala group and Arrhenius attempted to eliminate astrophysics from the scope of the prize in physics. Arrhenius, who wrote the proposal, seems to have feared that the establishment of new Nobel institutions would threaten his own Nobel Institute for Physical Chemistry. [...] Previously, only Arrhenius's institute had been supported in this way, but during the war Carlheim-Gyllensköld suggested a department for cosmical physics that was later followed by detailed plans for an astrophysics centre²⁹'³⁰.

It goes on. More briefly, Swedish physicists' attempts to boost their own fields led to politicking within the Nobel Prize in Physics committee that shortchanged astrophysics, atmospheric physics, and physics that was too theoretical.