All of mouseking's Comments + Replies

To put it as simply as I could, LessWrongers like to quantify stuff. A more specific instance of this is the fact that, since this website started off as the brainchild of an AI researcher, the prevalent intellectual trends will be those with applicability in AI research. Computers work easily with quantifiable data. As such, if you want to instill human morality into an AI, chances are you'll at least consider conceptualizing morality in utilitarian terms.

The confluence of a number of ideas. Cox's theorem shows that degree of belief can be expressed as probabilities. The VNM theorem shows that preferences can be expressed as numbers (up to an additive constant), usually called utilities. Consequentialism, the idea that actions are to be judged by their consequences, is pretty much taken as axiomatic. Combining these gives the conclusion that the rational action to take in any situation is the one that maximises the resulting expected utility. Your morality is your utility function: your beliefs about how people should live are preferences about they should live. Add the idea of actually being convinced by arguments (except arguments of the form "this conclusion is absurd, therefore there is likely to be something wrong with the argument", which are merely the absurdity heuristic) and you get LessWrong utilitarianism.

In general this site focuses on the friendly AI problem, a nihilistic or a hedonistic AI might not be friendly to humans. The notion of an existentialist AI seems to be largely unexplored as far as I know.

To the extent that lesswrong has an official ethical system, that system is definitely not utilitarianism.

Well, welcome to LessWrong anyway! Glad to you decided to join the conversation, talkative or not.