Was pissed off when Harry disagreed with his Yule speech, and apparently not just because Harry said so in public. Rather, he seems to really cares that Harry agree with him about it.

I think the Yule speech was largely to set up the wizarding world to take Harry as their Dear Leader. Having Harry argue against it was not what Quirrell had in mind.

Quirrell:

It should have been obvious even to you that you should have stayed silent, and consulted with me first, not spoken your worries before the crowd!”

I think he wants Harry to be the Dark Lord too, so that in the end he can take over his body like Quirrell's and rule as Dark Lord Harry, when Harry seems to defeat Voldemort.