What I am concerned about is that even upon being exposed to the logic of non-zero-sum games, some people remain remarkably selfish and/or cynical. Is there any way to deliberately increase their altruism and/or idealism without resorting to mysticism or religion?

Interesting point. I have actually noticed that among my friends the religious ones are also the less selfish and cynical ones (even though that wasn't what you were saying). However, they are less selfish even in zero-sum games, so there's a point against resorting to mysticism or religion.

As for increasing their altruism, I found that learning of the Iterated Prisoner's Dilemma and the tit-for-tat strategy make a marked difference for me. Maybe this could generalise to "learn rationality".