Why did you decide to keep a reading journal in the first place? This might help you come to an answer.

I have taken notes on fiction only a few times when reading for pleasure - mostly in reading sf when some new idea or speculation made me wonder about its further implications and I jotted it down for discussion with other like minded people. That worked fine and I still do it once in a while. I also took notes once or twice when reading fiction as part of a reading group, again with the idea of highlighting discussion points that interested me. But I felt that taking notes detracted from the actual reading and enjoying part of reading - discussion wasn't any different without notes.