But Harry's memory didn't include Voldemort casting Avada Kedavra on Harry. That memory is neutral WRT the "rebounded Avada Kedavra" hypothesis.

Also, no part of a rebuttal to your comment, but re-reading the scene, what's with this line?:

And the boy in the crib saw it, the eyes, those two crimson eyes, seeming to glow bright red, to blaze like miniature suns, filling Harry's whole vision as they locked to his own -