Why do you like "I won't kill you, but I don't have to save you" so much?

For a start because it is slightly less insane than actively saving the supervillians.

Sounds like him just rationalising his way around his no killing rule, which depends on a false action/inaction distinction.

I reject your claim. There is a difference between killing and not saving. One of the most obvious differences is one that you seem to have forgotten since your first paragraph:

but its more complex in the context of the dark knight as its about restoring trust in the rule of law in Gotham and Batman dispensing vigilante justice diminishes that, so there may be a net utility loss.

Not making superhero-level interventions to save supervillains is not vigilante justice. As such the supposed "net utility loss" from vigilante justice does not occur. The same applies to many (albeit not quite all) of the ethical considerations with respect to murder. (I refer to things like "Don't murder even when you think it is the best thing to do because you are likely to incorrectly calculate probabilities of getting caught, etc.)

Batman doesn't have a "you have to save everyone" rule. He does have a 'no killing' rule. It's easy to tell the difference between the two decisions. Anyone claiming that not saving someone is the same as murdering them is just wrong.

I also declare, among other things, that going to Africa and hunting Africans with a sniper rifle is ethically distinguishable to not donating to charities that will save the same lives. It would be fair to say that one of the reasons that the quote appeals so much is that it rejects the obnoxious "not saving is murder" fallacy that springs up around here all too often.