And it’s unlikely that the human species has reached the end of moral development.

...

IAnissimov: What is metaethics and how is it relevant to Friendly AI theory?

Metaethics goes one level deeper. What do terms like ‘good’ and ‘right’ even mean? Do moral facts exist, or is it all relative? Is there such a thing as moral progress? These questions are relevant to friendliness content because presumably, if moral facts exist, we would want an AI to respect them. Even if moral facts do not exist, our moral attitudes are part of what we value, and that is relevant to friendliness content theory.

So we are sure we don't want to fix things into place but we are not sure anything like moral progress exists? Isn't the distinction between moral change and moral progress the notion that the latter is normative while the former may not be? Also is the use of moral development here a synonym of moral progress or is it closer to my use of moral change or does it have a different meaning altogether?

Does the answer to the bolded question seem to be "yes" in Luke's opinion, but he also considers the implications of a "no" answer to carry significant enough impact to be included on the list? Note I don't find it likley the process of moral change or development has ended, I simply question that is should be ought when it comes to these matters.

In a way I am asking which part of the current or planned metaethics investigations has covered this? It seems to me a very basic thing to ask ourselves.