The confluence of a number of ideas.
Cox's theorem shows that degree of belief can be expressed as probabilities.
The VNM theorem shows that preferences can be expressed as numbers (up to an additive constant), usually called utilities.
Consequentialism, the idea that actions are to be judged by their consequences, is pretty much taken as axiomatic.
Combining these gives the conclusion that the rational action to take in any situation is the one that maximises the resulting expected utility.
Your morality is your utility function: your beliefs about how people should live are preferences about they should live.
Add the idea of actually being convinced by arguments (except arguments of the form "this conclusion is absurd, therefore there is likely to be something wrong with the argument", which are merely the absurdity heuristic) and you get LessWrong utilitarianism.
Utilitarianism is more than just maximizing expected utility, it's maximizing the world's expected utility. Rationality, in the economic or decision-theoretic sense, is not synonymous with utilitarianism.
Previous open thread
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.
Notes for future OT posters:
1. Please add the 'open_thread' tag.
2. Check if there is an active Open Thread before posting a new one.
3. Open Threads should be posted in Discussion, and not Main.
4. Open Threads should start on Monday, and end on Sunday.