Related: The Reversal Test: Eliminating Status Quo Biases in Applied Ethics, by Nick Bostrom and Toby Ord.

The great majority of those who judge increases to intelligence to be worse than the status quo would also judge decreases to be worse than the status quo. But this puts them in the rather odd position of maintaining that the net value for society provided by our current level of intelligence is at a local optimum, with small changes in either direction producing something worse. We can then ask for an explanation of why this should be thought to be so. If no sufficient reason is provided, our suspicion that the original judgment was influenced by status quo bias is corroborated.

[. . .]

The rationale of the Reversal Test is simple: if a continuous parameter admits of a wide range of possible values, only a tiny subset of which can be local optima, then it is prima facie implausible that the actual value of that parameter should just happen to be at one of these rare local optima [. . .] the burden of proof shifts to those who maintain that some actual parameter is at such a local optimum: they need to provide some good reason for supposing that it is so.

Obviously, the Reversal Test does not show that preferring the status quo is always unjustified. In many cases, it is possible to meet the challenge posed by the Reversal Test [. . .] Let us examine some of the possible ways [. . .]

The Argument from Evolutionary Adaptation [. . .]

The Argument from Transition Costs [. . .]

The Argument from Risk [. . .]

The Argument from Person-Affecting Ethics