We innately know a lot about "the physical environment", but not a lot about math (because the math abilities of your ancestors have not been under selection pressure for any evolutionary-significant length of time). Although it is true that neuroscience has found brain circuitry specialized for understanding non-negative integers, it remains the fact that much more of an educated person's knowledge about math must be acquired through deliberate practice, which is slow, than his or her knowledge about stacking boxes or replacing an alternator in a car.
In summary, there is no analog in math of your experience of having your knowledge about the physical environment unlock just because you chose to pay more attention to the details of your the physical environment.
We all have an innate drive to understand (i.e., curiosity) and also an innate drive to try to win arguments. Unlocking those 2 motivations is the closest analog I can think of to of your experience in the factory of getting a skill or innate ability to unlock, but before you can wield math towards profitable ends, you must spend many hundreds of hours reading about math and doing math. The 2 motivations I just described merely make it possible for you to put in those many hundreds of hours with less use of willpower or discipline.
I recently started working in a setting making physical products, where I discovered that I had a whole nother section of my brain purposed for physical interactions between things and when things might fall on other things or melt or fall apart or shatter or scar or any number of other things. Unfortunately, because of my inexperience, this is a part of my brain I had to be prompted to use. I had to be asked directly: "What do you think will happen as a result of that?" in order to actually use that part of my brain.
I can't help but feel that the same thing must be going on with my math ability.
I want to apply my math knowledge to the world so I can actually get some use out of it, but I have no idea how to go about doing that.
Wat do?