I mostly meant that "free rider" isn't a problem in altruism

Actually, it is. The problem with 'free riding' is not that it's somehow unfair to the people who are picking up the slack, it's that it distorts behavior. You don't want to give money to beggars if this just incents more people to beg and begging is a horrible job - and this is true even if you're altruistic towards people who might beg. You'll need to find a way to give money that doesn't have these bad consequences, even if that means expending some resources.