I don't really have well thought-out ideas on elitism, but I think there are places like MIT that have high standards that everybody accepts as normal, and I'm not sure of what distinguishes situations where the high standards are accepted, and situations where those standards would give rise to accusations of "elitism".

Overall I don't consider "elitism" a very useful word because it's vague and can describe many different things: "my group has more qualities than other groups", "you should grand me special respect just because of this group I belong to", "some people are more skilled or more gifted than others", etc. - add to that a general negative connotation, and it seems like a multi-purpose boo light, along with "fascist", "anti-American", "defeatist", "terrorist", "cult", "fanatic", etc.

(have you read the sequence on words? It's very relevant here, especially the bit on sneaking in connotations )