I don't think it's that my sleep quality is bad: I fall asleep quickly (usually within 10 minutes or so of turning the light out) and don't often wake up during the night. I don't usually remember dreams or have a sense that my sleep was disturbed.

For example in sleep apnea it's possible to wake up dozens of times per night and not remember anything about it. These people also fall asleep quickly because they're exhausted. I'm not saying that you have sleep apnea, just saying that not remembering waking up or falling asleep quickly aren't great indicators of healthy sleep.

As a side note some sleep medications (benzodiazepines, z-drugs) also impair short term memory, and people who use these might not remember if they wake up in the middle of the night, which makes these drugs even more popular.