I think that, in the first few chapters, Harry did not give enough credence to the hypothesis that he was simply insane and hallucinating. I think, given the observations he had at the time (his mom claimed her sister was a witch; he got a letter implying the same; a woman levitated his dad and turned into a cat), he should have at least seriously considered it. Certainly those pieces of information are some evidence for magic, but considering what that hypothesis entails — existing scientific knowledge about physics (even at the level of abstraction that we experience directly) is so completely wrong that it's actually possible to make the universe understand human words or intentions, or there's this incredibly advanced technology that looks like it's violating the laws of physics, and it's existed for thousands of years and apparently everyone has forgotten how it works — I think an honest rationalist would have to look into the "I'm cuckoo" hypothesis.