You might want to read this PDF: http://www.occampress.com/grades/grmath.pdf

If you read it you'll see that you do not have to be a child prodigy, don't have to start early on, don't need a good formal education and do not need to appear particular smart to work wonders (at least in mathematics). There are dozens of examples.

• “[Isaac] Newton (1642-1727) ... was educated at local schools of low educational standards and as a youth showed no special flair, except for an interest in mechanical devices.”
• “Before Newton and Leibniz, the man who did most to introduce analytical methods in the calculus was John Wallis (1616-1703). ... he did not begin to learn mathematics until he was about twenty — his university education at Cambridge was devoted to theology...”
• Gottfried Wilhelm Leibniz (1646-1716), co-discoverer of the calculus and a pioneer logician, “knew almost no mathematics up to 1672 [when he was 26].”
• “Hermann Günther Grassmann (1809-77) [discoverer of vector algebra], ... showed no talent for mathematics as a youth and ... had no university education in mathematics, but later became a teacher of mathematics in the gymnasium (high school) at Stettin, Germany...”
• Francois Viète (1540-1603), who laid the foundations for the algebraic approach to geometry, spent most of his life in high public office; “it was only during the time he had free from official duties that he was able to devote himself to mathematics.”
• Gerard Desargues (1591-1661), one of the pioneers of projective geometry, was self-educated. Yet “Desargues was one of the most original mathematicians in a century rich in genius.”
• Gottfried Wilhelm Leibniz (1646-1716), co-discoverer of the calculus, was selftaught in mathematics.
• James Bernoulli (1655-1705), a member of the remarkable Bernoulli family that made so many contributions to the early development of the calculus, was self-taught.

I like learning new things and love reading but any new ideas require a ton of thought and re-reading.

Don't be intimidated by most of the people on LW. They largely are not smarter but much more educated. A lot of the people here on LW have the best formal education in the world so they came across a lot of concepts before. Things are not obvious to them either, they just learnt to accept them. Is 1 + 1 = 2 really obvious? No! The last page of Russel and Whitehead's proof that 1+1=2 could be found on page 378 of the Principia Mathematica. The complete proof of 2 + 2 = 4 involves 2,452 subtheorems in a total of 25,933 steps!