These seem to be the relevant quotes:

"For some reason or other," said the amused voice of Professor Quirrell, "it seems that the scion of Malfoy is able to cast surprisingly strong magic for a first-year student. Due to the purity of his blood, of course. Certainly the good Lord Malfoy would not have openly flouted the underage magic laws by arranging for his son to receive a wand before his acceptance into Hogwarts."

and

Only there was a reason why they usually didn't bother giving wands to nine-year-olds. Age counted too, it wasn't just how long you'd held a wand. Granger's birthday had been only a few days into the year, when Harry had bought her that pouch. That meant she was twelve now, that she'd been twelve almost since the start of Hogwarts. And the truth was, Draco hadn't been practicing much outside of class, probably not nearly as much as Hermione Granger of Ravenclaw. Draco hadn't thought he needed any more practice to stay ahead...

-both hpmor ch.78

So from this it seems magic power increases with age, spells cast and time since first getting your wand (though the third could simply be due to the second)

So the reason Harry can only just now cast second year spells, is that he has only recently become sufficiently powerful. His partial transfiguration and patronous v2.0 don't actually require a lot of spell power they only require you to do clever things.