Was Kant an analytic philosopher? I can't remember, but thinking in terms of your actions as being the standard for a "categorical imperative" followed by yourself in all situations as well as by all moral beings, the effect of giving the mugger the money is more than $5. If you give him the money once he'll be able to keep on demanding it from you as well as from other rationalists. Hence the effect will be not $5 but all of your (plural) money, a harm which might be in a significant enough ratio to the deaths of all those people to warrant not giving him the money.

I think we have assume that, although this sounds awfully like that quote about "a million deaths are a statistic", the cost of additional deaths decreases. I'm not really sure how to justify that though.