You may have to correct my history but weren't modern composition techniques pretty unpopular right away. It isn't like academic composers have been building off each other for decades with few listening and so now their music isn't intelligible. Rather the introduction of atonality, the 'liberation of dissonance' and moving off the diatonic scale were very rapid changes to music which were very alienating. This prompted Adorno to say things like

The dissonances which horrify the public testify to their own condition and it is for this reason alone that they are unbearable for them.

I'm curious what you think of his position, actually.

So I'm not sure inferential distance is the right metaphor. It seems to me that while the uninstructed listener may not understand the works of modern academics, they likely didn't understand the works of Beethoven either but were still able to enjoy them for emotions they evoked. Contemporary music evokes emotion and while I don't know a lot about it I can enjoy it (partly, I think, because I've learned to enjoy the more avant garde end of pop music) but the emotions contemporary evoke tend to be more complex, and darker or at least bittersweet. I don't feel at home listened to contemporary music and I think thats the experience created by dissonance and what a lot of people recoil from.

Where does like, John Adams fit into this? He seems fairly accessible to the uninstructed.