Good point Bacon. I've been wondering where the implicit assumption that rational agents have an altruistic agenda came from. The assumption seems to permeate a rather large number of posts.

When Omega offers to save lives, why do I care? To be perfectly honest, my own utility function suggests that those extra billions are a liability to my interests.

When I realise that my altruistic notions are in conflict with my instinctive drive for status and influence, why do I "need to move in the direction of joining groups more easily, even in the face of annoyances and apparent unresponsiveness"? If anything it seems somewhat more rational to acknowledge the drive for status and self-interest as the key component and satisfy those criteria more effectively.

This isn't to say I don't have an altruistic agenda that I pursue. It is just that I don't see that agenda itself as 'rational' at all. It is somewhere between merely arbitrary and 'slightly irrational'.

With that caveat, this summary and plenty of the posts contained within are damn useful!

"...then there's the idea that rationalists should be able to (a) solve group coordination problems, (b) care a lot about other people and (c) win..."

Why should rationalists necessarily care a lot about other people? If we are to avoid circular altruism and the nefarious effects of other-optimizing, the best amount of caring might be less than "a lot."

Additionally, caring about other people in the sense of seeking emotional gratification primarily in tribe-like social rituals may be truly inimical to dedicating one's life to theoretical physics, math, or any other far-thinking discipline.

Caring about other people may entail involvement in politics, and local politics can be just as mind-killing as national politics.

It might be prudent to avoid associating rationality with particular people or social institutions.

There's always the risk that particular instances of rationality will result in disaster, or that Bad Guys will be painstakingly rational, and in the early stages, wouldn't want to suffer the fate of religions, which often take reputation hits when their followers do nasty things.

Rationality could be advertised as a morally neutral instrumental value, i.e., Better Living Through Rationality.

On the other hand, we could sell rationality as a tool for atheists, drug policy activists, and stockbrokers, and publicly associate with their successes.

I would venture that emotivism can be a way of setting up short-run incentives for the achievement of sub-goals. If we think "Bayesian insights are good," we can derive some psychological satisfaction from things which, in themselves, do not have direct personal consequences.

By attaching "goodness" to things too far outside our feedback loops, like "ending hunger," we get things like counterproductive aid spending. By attaching "goodness" too strongly to subgoals close to individual feedback loops, like "publishing papers," we get a flood of inconsequential academic articles at the expense of general knowledge.