For what it's worth, you've echoed my experience precisely.
It's easy and good to talk in one-on-one or small groups of people. It's also easy for me to speak publicly or in front of an audience. But interaction in strictly social settings - parties, bars, clubs - is almost painful. I don't know how to begin. The impression is just that of trying to start climbing a wall that admits no footholds.
I'm pretty certain I don't fall very far on the Asperger's spectrum. In particular, I can read body language and subtext pretty well, and I'm considerate of others to a degree I haven't seen in people I know who actually have Asperger's. I suspect that, as others have said, we're conflating distinct causes under a single banner.
Followup to: Do you have High-Functioning Asperger's Syndrome?
LW reader Madbadger uses the metaphor of a GPU and a CPU in a desktop system to think about people with Asperger's Syndrome: general intelligence is like a CPU, being universal but only mediocre at any particular task, whereas the "social coprocessor" brainware in a Neurotypical brain is like a GPU: highly specialized but great at what it does. Neurotypical people are like computers with measly Pentium IV processors, but expensive Radeon HD 4890 GPUs. A High-functioning AS person is an Intel Core i7 Extreme Edition - with on-board graphics!
This analogy also covers the spectrum view of social/empathic abilities, you can think about having a weaker social coprocessor than average if you have some of the tendencies of AS but not others. You can even think of your score on the AQ Test as being like the Tom's Hardware Rating of your Coprocessor. (Lower numbers are better!).
If you lack that powerful social coprocessor, what can you do? Well, you'll have to run your social interactions "in software", i.e. explicitly reason through the complex human social game that most people play without ever really understanding. There are several tricks that a High-functioning AS person can use in this situation: