The problem is that he did not - he treated them as a passive audience without any consideration of how they view him. So now some of them have reached the same conclusion as Lucius, and think he is a case of bodysnatching. Possession is a real possibility in the universe he inhabits, and he is showing all the signs. That is quite likely to get him killed by people with the best of intentions. At best, I am expecting kidnapping attempts aimed at extracting voldemort from his host. Also, Harry really should listen to Malfoy. Scaring Lucius is not a good idea.