Canon!Harry was especially vulnerable to Dementors, which raises the possibility that the Dementor is there to influence Harry's personality, increase Quirrell's hold over him, and shape him into what Quirrell wants him to be. Dumbledore described Harry as having a mind like a Dark Lord but with love which makes him non-evil, Dementors suck out love & happiness. And Dumbledore is supposed to be the one protecting him, so if anything bad happens to him....