What is your pond? I think more details would help.
My most relevant pond now is that I play Magic: the Gathering competitively, with a focus on deckbuilding. The online Magic community has grown a lot since 2001 and I'm not too thrilled at my chances of making a name for myself in it, at least not without doing something dramatically different than I have been. I either need a mentor who's better than I am (Zvi, of the NY Less Wrong meetup group, turned me down), or I need to start working a lot harder and spending a lot more money on the problem of trying to qualify for the Pro Tour. (Not just on cards, but on travel, too.)
I don't have a job. When I start looking at job postings, I freak out, and I also have no confidence in my ability to get and keep a job in the current economy. My meatspace social life is kind of crap, too; I live with my parents, who support me, and rarely see anyone else in a social context. Needless to say, being with my parents makes me the small fish, not the big fish. I also used to go to the NY meetup group but it's such a pain in the neck to take the bus to NYC and I feel inadequate next to these people with well-paying jobs and/or advanced degrees.
I could try to get some respect as a productive member of a raiding guild in some MMO or other; I'd only have to meet a threshold of competence instead of competing against other people, so it would be less stressful. I do have a pretty good healer in SWTOR...
I've always been very good at math. In my (small) high school, I was always the best math student; in college, I still felt like I was among the top math students in any given math class I took, although I was only a math minor, not a math major. If I tried to study math in graduate school, I don't think "lack of talent" would be an obstacle to getting an advanced degree... but how much room is there in the world of mathematics for someone who's probably closer to the 1 in 1000 level than the 1 in 100,000 level? And it's not like I like taking classes...
I'd agree with Lumifer and jamesf that it seems like it'd be best to do what you can to overcome the ugh field surrounding getting a job. Anecdotally, from my experience and others, having a job can do wonders for your self-esteem and general outlook on life. And it's also a reason to get out of the house and meet new people!
On that note, do you have any interest in computer programming? Programming ability seems to be pretty correlated with mathematical ability, at least to the degree that anybody at the 1 in 1000 mathematical ability level should be a...
This is a thread where people can ask questions that they would ordinarily feel embarrassed for not knowing the answer to. The previous "stupid" questions thread went to over 800 comments in two and a half weeks, so I think it's time for a new one.