That makes sense.. but immediately afterward both Harry and McGonnagal thought it was unusual how bright the remembrall shined; neither seemed to think it was solely due to the use of the Time Turner:
"The Remembrall was glowing bright red in his hand, blazing like a miniature sun that cast shadows on the ground in broad daylight."
and
""More importantly, why did the Remembrall go off like that?" Harry said. "Does it mean I've been Obliviated?"
"That puzzles me as well," Professor McGonagall said slowly. "If it were that simple, I would think that the courts would use Remembralls, and they do not. I shall look into it, Mr. Potter.""
And then of course she doesn't. Perhaps the courts simply don't use Remembralls because they would never definitively prove obliviation - only that something was forgotten. Harry's remembrall "blazing like a miniature sun" may be due to an overwhelmingly large obliviation - like an entire life as a Dark Lord? Obliviating a day or week may just produce a normal glow...
I think the most interesting part of this chapter (82) is another two clues about the Harry's Dark Side/Voldemort connection:
"Why was there a part of him that seemed to get angry at the old wizard beyond reason, lashing out at him harder than Harry had ever hit anyone, without thought of moderation once the rage had been raised, only to quiet as soon as Harry left his presence?"
Hmm, Harry's dark side mysteriously hates Dumbledore but doesn't remember why..? This is just one more clue that his dark side is an obliviated Voldemort or a horcrux - Voldemort's memories influence his dark thinking even if he doesn't remember why.
Also,
" 'Step aside, foolish woman, if you have any sense in you at all -' An awful chill came over Harry as he spoke those words from his own lips, but he shook it off and continued."
This could just be a creepy thing to hear yourself say about your mother, but could it be even more creepy if you realized you'd already heard yourself say it? Thinking back to the Remembrall incident, it's likely Harry has memories of Voldemort that are slowly coming out...
Also, Fawkes did have some connection with Harry after he returned from Azkaban - he asked Harry to help him destroy the place, and it's clear he asked the same of Dumbledore (perhaps many times).
So he's not totally off the mark in thinking that Fawkes would come to him - in that case he'd be agreeing to do what Fawkes wants, instead of restraining him like Dumbledore usually does. I bet if Harry called to Fawkes with a wish to go to Azkaban and destroy it, Fawkes would be there to take him in a heartbeat.