Neurological similarity: Most male orgasms result from stimulation of the penis. Most female orgasms result from stimulation of the homologous organ, the clitoris.
Anatomical difference: Standard sexual intercourse stimulates the penis very effectively, but stimulates the clitoris only moderately.
In a non-iterated interaction, the male partner has less incentive to perform activities likely to result in female orgasm. No brain difference is required to explain this effect.
Dr. Aaron Sell recently wrote an interesting piece about political incentives and shoddy statistical work in science. In particular, he was highly critical of the much-publicized Conley article in the Journal of Personality and Social Psychology which attempted to demonstrate that "large gender differences... [in willingness to engage in] casual sex [have] more to do with perceived personality characteristics of the female versus male proposers than with gender differences."
[[The full article can be found here. Please note - I have deliberately chosen to omit quoting any material which Less Wrongers may find mindkilling. It would be nice if we could keep it that way.]]
He points out that Conley's results were obtained by hypothetical pseudo-experiments, and spurious controls. For example, when Conley found that men were more willing to sleep with their friends than women, she made this difference "evaporate" by:
Yet for bottom line reasons, we now find Eagly and Woods reading Conley uncritically, and even claiming that she has overturned Clark and Hatfield's classic (and truly experimentally based) article demonstrating that women are radically less willing than men to have casual sex.
Sell concludes:
While empiricism is great, I have long believed that the social and organisational structures in which science is practised makes it especially vulnerable to political capture, so this plays right into my biases. Am I missing something important?