I don't care about colloquial usage, sorry. e You should car, because no-one can make valid arguments based on arbitrary definitions. I can't prove angels exist, by redefining "angel" to mean what "seagull" means. How can you tell when a redefinition is arbitrary (since there are legitmate redefinitions)? Too much departure from colloquial usage.

Eliezer has a convincing explanation of why wishes are intertwined with morality ("there is no safe wish smaller than an entire human morality").

"Intertwined with" does not mean "the same as".

I am not convinced by the explanation. It also applies ot non-moral prefrences. If I have a lower priority non moral prefence to eat tasty food, and a higher priority preference to stay slim, I need to consider my higher priority preferece when wishing for yummy ice cream.

To be sure, an agent capable of acting morally will have morality among their higher priority preferences -- it has to be among the higher order preferences, becuase it has to override other preferences for the agent to act morally. Therefore, when they scan their higher prioriuty prefences, they will happen to encounter their moral preferences. But that does not mean any preference is necessarily a moral preference. And their moral prefences override other preferences which are therefore non-moral, or at least less moral.

Therefore morality si a subset of prefences, as common sense maintained all along.

I've made that switch so long ago that I've forgotten how to think otherwise.

IMO, it is better to keep ones options open.