Agreed that individual humans have differing values. However I believe there's more to say about human universal values than "we're competing with one another". Here are two pieces of evidence:
(1) Almost all human cultures have what we recognize as art. So far as I know, chimpanzee cultures do not. (Although a quick google search tells me that chimps who live in human cultures do create art.) So art, broadly defined, is universal; but individual humans have more specific tastes. I expect humans and chimpanzees have senses of justice (differing from individual to individual).
(2) Individuals can change their values through moral arguments and education. If I try, through moral argument, to convince someone of my idea of justice, it's because I feel that my idea of justice has at least a somewhat universal appeal. And, with sufficient resources, you could become a musician and gain a richer appreciation of music. One could imagine an extrapolation process under which individuals' values converge, at least some of the time.
However:
And when we remove some human values- predation, cruelty, rape, domination- from the list because their time has passed, it is not clear to me why the others remain.
I share the concern that I think you're raising here: For all we know, moral progress might not be pointing towards any particular destination. More research needs to be done on the mechanics of morality.
Mostly agreed.
And, with sufficient resources, you could become a musician and gain a richer appreciation of music.
This I'm not as sure about. I can become competent at playing an instrument and better at interpreting music, but I'm not sure I can rewrite my talents. If those are determined by a relatively inelastic part of my brain's configuration, then it seems likely that the pathway to becoming a Beethoven simply does not exist for me.
A better example might be heroic compassion- people who put themselves at risk without thinking to save others. The...
Taken from some old comments of mine that never did get a satisfactory answer.
1) One of the justifications for CEV was that extrapolating from an American in the 21st century and from Archimedes of Syracuse should give similar results. This seems to assume that change in human values over time is mostly "progress" rather than drift. Do we have any evidence for that, except saying that our modern values are "good" according to themselves, so whatever historical process led to them must have been "progress"?
2) How can anyone sincerely want to build an AI that fulfills anything except their own current, personal volition? If Eliezer wants the the AI to look at humanity and infer its best wishes for the future, why can't he task it with looking at himself and inferring his best idea to fulfill humanity's wishes? Why must this particular thing be spelled out in a document like CEV and not left to the mysterious magic of "intelligence", and what other such things are there?