I have had problems with that too. Usually if the point seems particularly insightful / important, I'll take out my phone to write down the note at that moment, usually as a text to myself. But that happens rarely. Normally, I just try to take notes when at the soonest convenient moment (e.g., at the end of my commute).

I think reading good summaries of the books before and after mitigates the downside of not taking notes a good amount, so I don't worry about this as much as I used to.