If you can't ever recall meeting someone who you concluded was smarter than you who wasn't good at X, and you didn't use any kind of objective criteria or evaluation system to reach that conclusion, then you're probably (consciously or otherwise) incorporating X into your definition of "smarter."

There's a self-promotion trap here -- you have an incentive to act like the things you're good at are the things that really matter, both because (1) that way you can credibly claim that you're at least as smart as most people, and (2) that way you can justify your decision to continue to focus on activities that you're good at, and which you probably enjoy.

I think the odds that you have fallen into this self-promotion trap are way higher than the odds for any of your other hypotheses.

If you haven't already, you may want to check out the theory of multiple intelligences and the theory of intelligence as information processing