OTOH it was Feynman who said something like ‘we don't know how to explain [something] to freshmen, therefore we don't really understand it yet’, and Einstein and Rutherford are alleged to have said similar things about explaining stuff to grandmothers and bartenders respectively.

He probably was using the word “understand” in a relatively narrow sense (after all he was the same person who said that no-one understood QM), and I agree with your general point, but certain people do overestimate how impossible it is to explain certain things in a way that can be understood by intelligent laymen (as done e.g. in Feynman's QED).