I know at least four people who started college by age 15. They're not "kid" geniuses anymore though -- the youngest is 16 and slowly going through college part-time, while the oldest is in his 30s and a full math professor at Arizona.

I don't know about the upbringing of the other three, but one attended a program where taking classes mutliple grade-levels ahead is the norm (though no-one else learned calculus in 3rd grade), and attended Canada/USA Mathcamp during the summers of his undergrad.

I second the Olympiads. Terry Tao famously represented Australia at the IMO at age 10, so he's definitely old enough.