It doesn't seem at all correct to say "average men treat women like they're silly, but rationalists don't do that!"

So we have a statement of the general form "X does Y, but we don't." This is the sort of statement that people are liable to say even when it's false, so we should heavily discount the weight that we would give to their personal opinion (because it might be biased.) Instead, we should hug the query -- gather more evidence (as unfiltered as we can -- statistics may be more unbiased than pure anecdote) or display that which has been gathered, instead of ignoring the possibility that the claim is factually true. If the claim is false, the evidence should tell us that as well.