perhaps the curse successfully detached baby-Harry's-soul [...], but the body was immediately made into a Horcrux... so Voldemort-soul-shard effectively inhabited that body. [...]

Further speculation: When Quirrellmort was lecturing in class about how he used to want to be a Dark Lord, but now finds that he really just wants to teach how to fight and such... perhaps that's a literal statement of fact?

This is awesome. Probably not where the story's actually headed, but it would create a cool Vader-Skywalker kind of relationship and explain what Voldemort is trying to accomplish with Harry. If he wants his Harry-shard to finish the job of becoming Dark Lord, then it makes sense to come to Hogwarts to be Harry's mentor (and to be disgusted by Harry's ambition to be a scientist).