Quirrel had seen Harry use /Diffendo/ on some trees, and later that the trees have been cut. He was unconscious (and in an extradimensional bag) when Harry had cut through the wall in Azkaban, and only saw a cut circle of wall. He may not have known that Harry had anything up his sleeve more complicated than a Cutting Charm; he certainly had no reason to believe that Harry could wordlessly transfigure the tip of his wand into well over a hundred feet of braided carbon nanotubes. Quirrelmort has never seen -- only Dumbledore, Hermoine, and Professor McGonagall have.

But that's really not the part that got him.

Quirrelmort had accepted the risk that Harry could have escaped, or killed everyone present, just as he accepted the possibility that the Unbreakable Vow wouldn't have been enough to stop Harry from destroying the world. If he were absolutely certain, he'd not have bothered with backup plans. He did not care about the deaths of the present Death Eaters, and losing his own body was merely a minor setback. It's Harry's ability to instantly and permanently incapacitate without letting Quirrelmort's spirit loose that made the threat serious. That's a problem Dumbledore was relying on an ancient and frighteningly powerful artifact to implement, and Quirrelmort's mode of thinking doesn't exactly encourage thinking of these matters..