Yes, it's super sad to let a little girl be tortured to death. But there is a cost large enough that it is not worth paying to prevent it, even if the cost is only in terms of mere cash, political capital, personal reputation as not being more fearsome than Fear itself, keeping important military secrets for the coming war secret, and the enmity of those you failed to lose to. That's the meaning of the phrase "Taboo Tradeoffs", it's that stating you kept Hermione out of Azkaban is not enough justification.

Of course, if he had counted the cost, he would have been an awful hypocrite. Recall what he said after Hermione rescued him from the Dementor:

I'll say that no matter what it ends up costing you to have kissed me, don't ever doubt for a second that it was the right thing to do.

At least he's holding himself to the same standard, even if it's a bad one.