Sexism can mean a whole bunch of different things. It's not a simple binary predicate: this is sexist, that isn't. In general, I mean a cluster of attitudes and actions that harm people based on their sex. Usually, its women being harmed, but definitely not always.

Affirmative action is, of course, an interesting case. On its face, it involves advantaging one group, which naturally comes at the expense of all other groups. So, of course it's sexism in one sense of the word. So why does anyone think it's fair? Because there are believed to be cognitive biases in play that prevent people from (for instance) selecting an equally qualified woman for a job (one day, I would like to write up a post on the evidence for this). The theory is that an explicit adjustment for these biases will result in treatment more like what there would have been if employers were unbiased. If this theory is correct, then in cases where we believe that there is such discrimination, maintaining the status quo would be sexism. Naturally, not all cases of affirmative action qualify for this.

As the discussion on The Bedrock Of Fairness shows, fairness can have many meanings. They frequently correspond almost exactly to meta-ethical stances (consequentialist, deontological, virtue ethics). I'm a consequentialist with regards to fairness (since I view it as merely a part of the whole system of ethics). And affirmative action is only justifiable under a consequentialist (or perhaps virtue ethics) framework of fairness -- and then only sometimes. I guess that is, as you say, one particular interpretation of fairness, but it's one that I would imagine is relatively common here, since consequentialist ethics are relatively popular on Less Wrong.