Good point. I checked by visualising a selection of people in my head asking this, male and female, with various characteristics. I had the same reaction to about equal numbers of men and women. Usually some something along the lines of "erm, can we add each other on facebook first?"

...Then again, I'm probably just particularly not-keen on giving people my phone number, and as such was reading the situation exclusively in terms of "which way of asking makes the certainty of me saying "no" less awkward?"

I think you were assuming a certain context and tone and approach that have been more closely associated with that phrasing in your personal experience, perhaps without realizing it.

Yes, but I guess the OP also had that kind of situation in mind.