It seems like your problem might be too in having much empathy for strangers, which (at least when dealing with panhandlers) shouldn't theoretically be too hard to deal with. If you cultivate a mindset of viewing beggars as parasites and degenerates you ought to be able to resist any impulses of sympathy which come up, especially since you already know that you're not helping them and many are in fact con-artists. It shouldn't really affect your other charity giving much either, since my understanding is that EA mostly focuses on giving medical aid to foreigners rather than dealing with poverty in areas with high costs of living like American cities.

On the other hand, it's very possible empathy isn't your real problem here. The feeling of gratitude (even faux-gratitude) and generosity from handing a few bucks to a hobo is a big rush; I certainly get more utility out of my spare change that way than I ever would buying junk food with it. If that's your issue than it might be smart to do what you're doing now and poison the good feeling by re-framing it as something shameful.