Dr_Manhattan comments on I've had it with those dark rumours about our culture rigorously suppressing opinions - Less Wrong Discussion
You are viewing a comment permalink. View the original post to see all comments and the full post content.
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (857)
The LW version:
Friendliness of AGI is impossible; this is because Coherent Extrapolated Volition is impossible; our volitions are in part determined by opposing others' so any extrapolation will produce a contradiction (a la once disease is gone, food is plentiful, hangovers don't exist, and you can have sex with anyone you like, the only thing that Palestinians and Israelis care about is denying the others' desires). Any optimisation process applied to human desires will necessarily make things subjectively and objectively worse. We are, in effect, falling down stairs at the speed of our optimising, and more optimisation of any kind will only make us fall faster and deadlier. There was no guarantee that the blind process of evolution would produce agents that form a consistent or positive-sum system, and indeed, it did not produce such agents. The future is unchangeably bleak and necessarily bad.
The Western memeplex version:
Humans are not essentially good-natured beings. The so-called moral progress of the recent era is no such thing - severe oppression of the minority has been swapped for a larger amount of minor oppression of the majority, abject slavery of some has been swapped for an equivalent amount of wage slavery for many, rape and physical coercion of sex has been reduced with a much larger increase in use of non-physical coercion with alcohol or money, and so on. We might alter the concentrations of suffering, spreading it out over many, but we can only increase the total levels of suffering because a large part of how we actually feel content or happy is based on relatives - that is, the pursuit of happiness ruins the world.
The seeds of our own hellish existences are planted within us from birth, they are irrevocable, and the only thing we can do about it is miserably fail to realise their deadly potential. Any kind of hopeful attitude, any kind of optimisation, any attempt to improve the world, is a guaranteed net negative because that is just how humans are.
Self-reported happiness data seems to not agree with this, unless Swedes are keeping some tortured children in the basement.
If you took areas of low happiness and made them happy, Swedish self-reporting of happiness would go down, is the gist of this idea.
The capital of Sweden is Omelas? Hmm...