"'avoid your coworkers' might be bad advice for an aspiring philanthropist."

I don't think you should avoid your coworkers. That's definitely bad advice. Not only are there the potential advantages you describe, they're probably really interesting people! I'm saying you should try to have a lot of different influences, and not let your coworkers form too much of your social circle.

(Which is also a good strategy given that you may may lose your job, and don't want to lose all your friends with it.)