One thing to remember is that retrieval is easier now than it has been for most of the history note-taking systems have been developed. Easy note entry should be a priority. I've kept a diary-like miscellaneous notes file where I just append a time-stamped entry of whatever any time I want to make note of something for over 10 years now. Currently there's 4.7M of text. (The whole file is occasionally too large to work with so I've split it to a recent notes and archived notes halves, and append the recent notes to the archived notes every few months.) I don't actually retrieve things from there very often, it's mostly for general concentration-aiding brain dumps. I haven't got a tagging scheme, though I probably should if I started taking notes I need to refer back to more.

Recently I've been inspired by the MineZone wiki book notes to write outline-based notes using Vim and the VimOutliner mode. Been doing this on some nonfiction books that promote various systems of behavior, mostly meditation and productivity books so far, distilling the core steps of the system to an outline and dropping the fluff.

So far I've just appended my notes into text files named after a topic, like productivity.otl, with the top-level headings being the names of the articles and an outline going below that. I might get started doing something similar with nonfiction that isn't about crisp, hierarchical behavior systems and for various math and theory stuff. I'm tempted to go back to Emacs and org-mode for the mathy needs for the inline LaTeX functionality.

I remember being inspired by Cosma Shalizi's notebooks page years ago as a model for a set of notes from disparate subjects.