Maybe that's because it's a false memory?

It is very likely. Also from Chapter 3, when McGonagall told Harry about his parents' murder:

And somewhere in the back of his mind was a small, small note of confusion, a sense of something wrong about that story; and it should have been a part of Harry's art to notice that tiny note, but he was distracted. For it is a sad rule that whenever you are most in need of your art as a rationalist, that is when you are most likely to forget it.