I am a man and my mother always told me not to stuff my pants pockets full becasue it makes me look fatter: wider hips. Her recommended solution was, somewhat ironically, a cowboyish man-purse. That may be the reason: women may care more about not looking fatter than men.