The treatment of this subject in Novum Organum might be illuminating.
From The Baconian Method:
104. But the intellect mustn’t be allowed •to jump—to fly—from particulars a long way up to axioms that are of almost the highest generality (such as the so-called ‘first principles’ of arts and of things) and then on the basis of them (taken as unshakable truths) •to ‘prove’ and thus secure middle axioms. That has been the practice up to now, because the intellect has a natural impetus to do that
...
Our only hope for good results in the sciences is for us to proceed thus: using a valid ladder, we move up gradually—not in leaps and bounds—from particulars to lower axioms, then to middle axioms, then up and up until at last we reach the most general axioms. ·The two ends of this ladder are relatively unimportant· because the lowest axioms are not much different from ·reports on· bare experience, while the highest and most general ones—or anyway the ones that we have now—are notional and abstract and without solid content. It’s the middle axioms that are true and solid and alive; they are the ones on which the affairs and fortunes of men depend.
...
the human intellect should be •supplied not with wings but rather •weighed down with lead, to keep it from leaping and flying. This hasn’t ever been done; when it is done we’ll be entitled to better hopes of the sciences.
105. For establishing axioms we have to devise a different form of induction from any that has been use up to now, and it should be used for proving and discovering not only so-called ‘first principles’ but also the lesser middle axioms— indeed all axioms.
There are small things someone might notice or come up with on their own*. Learning, the first time around, seems like it's usually not done that way.
*Like how to take a sequence of numbers (1, 4, 9), and come up with a polynomial equation that fits (f=x^2). Working out that there's a faster/algebra way to add consecutive numbers together is fairly straightforward (and probably anything else Gauss did is more impressive).
Maybe Euclid did? There's also some relevant philosophy of science from Einstein (a la plato.stanford.edu):
[Concept Names] Einstein’s most original contribution to twentieth-century philosophy of science lies elsewhere, in his distinction between what he termed “principle theories” and “constructive theories.”
[Explanation] This idea first found its way into print in a brief 1919 article in the Times of London (Einstein 1919). A constructive theory, as the name implies, provides a constructive model for the phenomena of interest. An example would be kinetic theory. A principle theory consists of a set of individually well-confirmed, high-level empirical generalizations, “which permit of precise formulation” (Einstein 1914, 749). Examples include the first and second laws of thermodynamics.
[Impact] Ultimate understanding requires a constructive theory, but often, says Einstein, progress in theory is impeded by premature attempts at developing constructive theories in the absence of sufficient constraints by means of which to narrow the range of possible constructive theories. It is the function of principle theories to provide such constraint, and progress is often best achieved by focusing first on the establishment of such principles. According to Einstein, that is how he achieved his breakthrough with the theory of relativity, which, he says, is a principle theory, its two principles being the relativity principle and the light principle.
In answering Teach's question "Learning Abstract Math from First Principles?", I wrote
This does match my personal experience, but I also have some mild OCD tendencies that tie in with this stuff, and so it may just be me. What do other LWers think?