Being the one who approaches has many advantages, but it comes with a cost -- one must learn to deal with rejection. There is a difference between knowing, generally, "my attractivity is probably average", and being specifically rejected by one specific sympathetic person who seemed to be interested just a while ago, but probably just wanted to talk.

Interpreting rejection as "these men are afraid of a honest / courageous woman" can help protect the ego. It could also be why the men said it -- to avoid an offense, a confrontation. (Women also say various things that don't make much sense, when they mean: "I don't consider you attractive.") I mean, if an extremely attractive woman would approach those men, a lot of them yould probably say yes and consider themselves lucky. (This is an experimentally testable prediction!)