I've seen feminists argue that ... Women are socially conditioned to defer to men ... Men are used to their privilege ...

I might agree, provided they're talking about group averages rather than about all women and all men -- this guy doesn't sound “used to his privilege” to me.

And if they're talking about group averages, I can't see their relevance to interactions between individuals. Suppose that blue-eyed people are taller in average than brown-eyed people, and everyone knows this. Suppose there are two people in a room, one with blue eyes and one with brown eyes. They need to take something off a shelf, and the taller one was the easier it would be to do that. It would be preposterous to say “the blue-eyed person should do that, and if she lets the brown-eyed person do that she's an asshole, as she could much more easily do that herself, given that brown-eyed people are shorter”, if the blue-eyed person happens to be 1.51 m (5') and the brown-eyed person happens to be 1.87 m (6'2'').