You've taken a sufficiently coherent political philosophy and pressed it into service as a moral philosophy, where it doesn't fit. The principle "do not harm" doesn't imply that you should (may?) give to charity because it makes you feel good. It only implies the converse, that you should give to charity if it makes you feel good.

But [Edit: one] purpose of a moral theory is to tell you when (if ever) to give to charity (and what charity to give to, etc.)