Coherent extrapolated volition (CEV) asks what humans would want, if they knew more - if their values reached reflective equilibrium.  (I don't want to deal with the problems of whether there are "human values" today; for the moment I'll consider the more-plausible idea that a single human who lived forever could get smarter and closer to reflective equilibrium over time.)

This is appealing because it seems compatible with moral progress (see e.g., Muehlhauser & Helm, "The singularity and machine ethics", in press).  Morality has been getting better over time, right?  And that's because we're getting smarter, and closer to reflective equilibrium as we revise our values in light of our increased understanding, right?

This view makes three claims:

  1. Morality has improved over time.
  2. Morality has improved as a result of reflection.
  3. This improvement brings us closer to equilibrium over time.

There can be no evidence for the first claim, and the evidence is against the second two claims.

There can be no evidence that morality has improved

Intuitively, we feel that morality has definitely improved over time.  We are so much better than those 17th-century barbarians who baited bears!

If you have such a strong belief, that must mean you have evidence for it.  That must mean you had some hypothesis, and the evidence could have gone either way; and the evidence went in such a way that it supported your hypothesis.

If you believe this, then in the comments below, please describe a scenario that could have happened, in which we would today believe that the values people had hundreds of years ago were superior to the values they have today.  Not a scenario in which some conservative sub-group could believe this; but a scenario in which society as a whole could believe it, and keep on believing it for a hundred years, without changing their values.

We can show that values have changed.  But we can have no evidence that that change is towards better values, whatever that means, rather than a value-neutral drift.  (I don't even know how to express coherently the idea that "values are getting better".)

If society agreed that another set of values were superior, they would adopt those values.  In fact, they would already have those values, prior to agreeing.  There can be no observed event supporting the hypothesis that morals have improved.  No matter how much you feel that they have improved, you cannot have empirical evidence, not even in principle.

Our values do not change as a result of reflection

Values, like biology and culture, evolve.  That doesn't mean getting "better" over time.  It means becoming more adaptive.

Take any moral advance you like.  Study its history, and you'll find people adopted it when it became economically advantageous to those in power do so.

Christianity

Do unto others as ye would have others do unto you.  Turn the other cheek.  Slaves, obey your masters.

The Roman Empire was not an empire; it was a forest fire.  It burned its way out from Italy and across the continent, using up each new land that it came to, stripping it of resources and funneling them to Rome.  When it burned its way out until pillaging the new area on its perimeter (increasing as R) could no longer support the area in its interior (increasing as R squared), it burned out and died.  It was not a sustainable economic model.  It relied on exploiting conquered peoples, and on suppressing them with armies built from the wealth acquired by conquering other people.  (Citation needed.  I'm not an expert on ancient Rome.)

With Christianity, you could exploit people without needing large armies to keep them in line.  Christianity was the technology that saved the eastern half of the Roman Empire and allowed its survival into the high middle ages; and that enabled the rise of Western European nations.  "Slaves, obey your masters" was an economic necessity.  (China had discovered Taoism and Buddhism centuries earlier.)

How did Christianity bring us closer to reflective equilibrium?  It didn't.  It brought us WAY out of reflective equilibrium.  The virtues expressed in the Iliad are pretty close to a reflective equilibrium.  When we introduced all this stuff about loving your enemy, the cognitive dissonance in Western ethics went up by orders of magnitude.  Even today, we've never gotten near to the level of equilibrium we had pre-Christianity.  Christianity, as promoted by Jesus, is pacifist, communist, non-materialist, unpatriotic, and anti-family.

Masculinity

Consider an even more significant moral advance:  The de-masculinization of the human race.  Until a few centuries ago, men were encouraged to fight each other pretty much as often as possible.  Excellence in combat was the single greatest virtue in most societies throughout all of history until the 20th century.  Beating up weaker boys not only wasn't bad; it was a kind of civic duty.

The destructive technology of the 19th and especially the 20th centuries required changes.  Armed conflict was no longer a cost-effective way to make money or resolve disputes.  Society had to be reprogrammed.  And as population density continued to rise, countries needed to be able to keep a million men in a single city without them turning on each other like rats in a cage.

Again, how did this bring us closer to an equilibrium?  It didn't, which is why confused men sometimes feel the need to have steam lodges and drum circles in the woods.

Slavery

Or take slavery.  Was the abolition of slavery in the US the result of reflective equilibrium?  The virtuous northern US, which happened to have a lot of textile mills and other industry requiring skilled labor, realized the monstrosity of the institution of slavery, which also happened to give the Southern states enough votes in the House of Representatives to implement tariffs and other economic laws that favored the production of raw materials over the manufacture of goods.

But, you say, the North also had plenty of farmers!  Yet these good Presbyterians were never tempted to have their apple orchards or their cranberry bogs tended by slaves.

That's because the northern US is cold.  It has a short growing season.  It's more economical to hire workers when you need them, than to keep slaves year-round.

The Civil War began just after mechanical reapers and other inventions began to make slavery uneconomical for more and more people, until they reached the tipping point at which the people with these devices could use anti-slavery as a weapon against their competitors.  If the War had been delayed fifteen years, the South would have been inundated with labor-saving farm devices that made keeping slaves cost more than it was worth, and would have suddenly seen the error of their ways and renounced slavery on their own.  And the North would have missed an opportunity to achieve hegemony and the high moral ground at the same time.

Values shift further from, not closer to, equilibrium over time

The world is not in equilibrium, and hopefully never will be.  The trend, historically, has been for cultural change to accelerate, bringing us farther from, not closer to, equilibrium.  (This trend may be reversing in the last several decades, a point which would require many additional posts to explore.)

Culture is the sort of thing that you can't predict, you can only simulate.  The only way to see how the world is going to develop is to wait for it to develop.

You may think that a super-intelligent AI can simulate this much, much faster than humans can.  And you would be right.  But the super-intelligent AI is part of the culture - you could say it is the culture - once it exists.  In the process of trying to reach reflective equilibrium, it will learn new things, and discover new possibilities, which will require it to re-evaluate all prior beliefs, taking it farther from, not closer to, equilibrium.  Is there any reason to think this process will converge, rather than diverge more and more, as it has for all of history?  If there is, it has not been articulated.

Values converge and reach equilbrium in the same way that evolution converges and reaches equilbrium:  Not at all.

New to LessWrong?

New Comment
111 comments, sorted by Click to highlight new comments since: Today at 8:47 AM
Some comments are truncated due to high volume. (⌘F to expand all)Change truncation settings

Upvoted for making an interesting general point. Downvoted for cartoonish history that reads like it's about some weird parallel universe. (A point by point criticism would require a comment of almost the same length, but if someone seriously disputes my claim, I can list half a dozen or so particularly bizarre claims.)

[-][anonymous]12y130

.

Well, where should I start? A few examples:

  • The Roman Empire reached its maximum extent under Trajan circa 100AD. (And even that was a fairly small increase relative to a century earlier under Augustus.) Signs of crisis started appearing only towards the end of the 2nd century, and Christianity started being officially tolerated only in the early 4th century. How these centuries of non-expansion before Christianity entered the political stage can be reconciled with the theory from the article is beyond me.

  • There is clear evidence that the fall of the Roman empire occasioned a huge fall in living standards throughout the former Empire, including its provinces that it supposedly only pillaged and exploited. (See The Fall of Rome and the End of Civilization by Bryan Ward-Perkins for a good recent overview.)

  • Ascribing the decline in masculinity to some mysterious "reprogramming" that is narrated in passive voice strikes as me as bizarrely incoherent.

  • Large cities are not a modern invention. In the largest cities of the ancient world, enormous numbers of men (certainly on the order of hundreds of thousands) lived packed together much more tightly than in modern cities.

... (read more)
-1[anonymous]12y
Monty Python:What have the Romans ever done for us :)
0PhilGoetz12y
I seriously dispute your claim.
5Vladimir_M12y
Please see my reply to Gabriel Duquette below.
-2drc500free12y
This came off as Meta-Contrarian Intellectual Hipster to me. Many religions are highly reflective, debating what actions adherents should follow to achieve ethical ideals and reach a moral state of being. Zen Buddhism debates paths to enlightenment for universal understanding and emotional control. Judaism debates ways of giving Tzedakah to best provide immediate relief, encourage self improvement, and minimize shame. Hinduism debates various moral causes and their karmic effects. Examining any of these highly reflective religions would at least address the hypothesis that "reflection does not change values." Christianity debates the divinity of Jesus' body, whether the material universe is fundamentally evil, and whether the Son of God is subordinate to God. The "ultimate shortcut" is a boon for recruitment, but has prevented any moral self-reflection. When your religion is stuck deciding whether or not the things you do to yourself and others impact your morality, you don't even have a framework to be reflective. Catholicism votes yes (what other belief system has to even spell that out?), but defines morality largely through avoiding and confessing to specific immoral behaviors, rather than debating different ways to achieve broader values. While we could contrast major religions and debate the impact of their reflective traditions, examining Christianity actually provides a very controlled environment to consider the hypothesis. Because it is naturally devoid of (and in many ways hostile to) reflection on its general social utility, we can contrast Christian life before and after the U.S. Constitution curtailed religious morality and established a highly reflective governing process for civic morality. The ethical and moral progress we've made since then - suffrage, emancipation, health, and quality of life - is clear on its face when we don't try to cherry-pick examples.
2prase12y
How this all relates to the comment you are replying to?

I think there is some evidence morality improved, one as a thought experiment, another as a real, factual evidence.

The thought experiment is considering that you would take someone from now, and teleport him back to another period of history (say the Dark Ages, or the Roman Empire), and in the other way, take someone from those time and teleport him to now. In both cases the person will be shocked, and need time to adapt. But I do think there will be a significant difference : the guy from now teleported to the Dark Ages will be horrified that they use torture and horrible punishments, the one teleported to the Roman Empire will be horrified about gladiator games and slavery. The guy from the past teleported to now will not be horrified by the lack of them, but surprised that we manage to do without them : how can you maintain civil peace and order without cruel punishments ? How can you feed people without slaves ? How can you entertain people without gladiators ? That's a fundamental point to me : many things which are unethical now were accepted in the past not because they were valued for themselves, but because they were perceived as the only solution to a worse problem. The s... (read more)

[-]atorm12y150

Traveler from the past: "What?! You let filthy lesser races marry your children? Gays aren't stoned in the streets? Why is that woman in a position of authority over men? Why is THAT woman not ashamed to be a single mother? Society has collapsed into a disgusting moral cesspool!"

If you want to see how people from the past might look at our "moral progress", ask your racist grandma.

1[anonymous]12y
.
2Eugine_Nier12y
Why doesn't that apply equally well to the traveler from our time?
-1[anonymous]12y
.
3TheOtherDave12y
Can you say more about why you believe that? Because it seems unlikely to me on the face of it. By way of analogy: a lot of people today do seem to believe that punishing criminals is right and proper for its own sake, and would oppose a penal reform that made prison less unpleasant even if such a reform were demonstrated to reduce crime. If transported to a future where criminals were forgiven for their crimes without punishment of any sort and treated with generosity and affection until they voluntarily chose to conform to social norms, I expect that many of those people would be appalled, even if that policy demonstrably worked as a way of keeping crime rates down. I expect that a lot of people from the Roman Empire would similarly be horrified by our attitudes towards slavery, towards religion, and much else.
1[anonymous]12y
.

If you believe this, then in the comments below, please describe a scenario that could have happened, in which we would today believe that the values people had hundreds of years ago were superior to the values they have today. Not a scenario in which some conservative sub-group could believe this; but a scenario in which society as a whole could believe it, and keep on believing it for a hundred years, without changing their values.

And yet for the majority of history most people believed that values were decaying. See, for example, the ancient Greek notion of the Ages of Man, the related Hindu concept of the Four Yugas, or the quote at the top of this article.

9Alejandro112y
A good point, but one could reply by distinguishing two situations, disambiguating the idea that "values are decaying": a) A society believes that the past generations were more virtuous, in the sense of behaving more in accordance to virtue (because of better intrinsic self-control, stronger social punishments for evildoers, or whatever reason), while still having in the present the same standards for virtue, only less observed. b) A society believes that the past generations had substantially different, and better, standards for virtue than the present one. The Ages of Man and similar historical decline beliefs seem to fit the first situation, while Phil seems to be arguing that the second one is impossible.

Our values do not change as a result of reflection

Values, like biology and culture, evolve. That doesn't mean getting "better" over time. It means becoming more adaptive.

Take any moral advance you like. Study its history, and you'll find people adopted it when it became economically advantageous to those in power do so.

The arguments here seem completely orthogonal to the heading they are supposed to support. In fact that seems to be representative of the whole post.

Yes, values are unstable over time. Go far enough back and there will not even be creatures with something that can be described as 'values'. The same could be expected if humanity somehow managed to evolve towards the Malthusian equilibrium that current evolutionary payoffs would reward. Our current values are completely unstable.

CEV isn't anything to do with predicting what humans would value in the future. It is about capturing the values we have right now and adjusting them only according to how we would want them to be adjusted if we had the time, power and resources to think it through. That doesn't mean simulating the future it means taking a closer look at inconsistencies and competing desires and... (read more)

I like the point that human values seem to be changing faster lately, so it doesn't look like they're approaching equilibrium. But the post seems to be making huge leaps of logic.

The Roman Empire was not an empire; it was a forest fire. It burned its way out from Italy and across the continent, using up each new land that it came to, stripping it of resources and funneling them to Rome. When it burned its way out until pillaging the new area on its perimeter (increasing as R) could no longer support the area in its interior (increasing as R squared), it burned out and died.

That sounds different from the standard story about Rome. Why do you believe that?

6PhilGoetz12y
It's one of many theories that I heard, and I thought it was the most plausible at the time, for reasons I no longer remember. Possibly because, unlike many theories about Rome, it wasn't designed to teach a moral lesson. Take it with a grain of salt. It is at best an oversimplification.

This theory has obvious intended moral lessons; I'd actually be very curious as to why you perceive it as morally-neutral. Is is that you see the primary lesson as so obvious ("pillaging isn't a sustainable economy") that it doesn't appear to be didactic? I wouldn't be surprised if the lessons were the whole point of the theory; I've heard it used before as an analogy to criticize the Soviet Union's political structure and the United States' economic structure (by two different people, naturally).

Now that I think about it, since entire schools of morality can be roughly summarized as "morals are the codes of conduct that make your civilization work well", I doubt it's even possible to come up with a theory explaining a civilization's collapse without that theory inherently expressing a moral lesson. Even "external factors destroyed it" could be interpreted as "you should be more paranoid than they were about dangerous external factors".

8timtyler12y
Re: Why is the roman empire like a forest fire? I expect that's an honest answer. I read it and thought: wow: it is so weird to hear an honest-sounding answer in response to the question of "Why do you believe that?" - rather than a story that serves as a defense of the original belief.
2gwern12y
This sounds like one of the narratives Tainter presents in Collapse of Complex Societies, FWIW, since it fits his overall theme: complex societies (like Rome) develop (militaristically expand) until the marginal return hits zero or goes negative (O(n^2) territory finally beats O(n) new area), and then they collapse (burned out and died).
0timtyler12y
Hmm. Human terminal values - warmth, freedom from pain, orgasms, sweet tastes, etc - are evolving along with the human genome - very slowly. Morality is a bit different. That is partly cultural - and evolves much faster (and is indeed getting better over time). It might look as though human terminal values change with culture too - but usually that is because culture produces environmental changes. The sterile catholic priest hasn't had their terminal values changed - rather their beliefs about the state of the world have been changed. Of course memes would love to be able to mess with human terminal values - but they are pretty wired in and seem rather challenging to mess with. Drugs may be the nearest thing.

"Slaves, obey your masters" was an economic necessity. (China had discovered Taoism and Buddhism centuries earlier.)

Confucianism would have been more appropriate.

1PhilGoetz12y
Good point. I think they all contribute toward the same end.

The Civil War began just after mechanical reapers and other inventions began to make slavery uneconomical for more and more people, until they reached the tipping point at which the people with these devices could use anti-slavery as a weapon against their competitors.

What about countries like Mexico, that both got rid of slavery before the technology you mention, and without using it much as a piece in power struggles? Some account of the widespread abolition of slavery might be written without reference to human values, but this isn't it and I don't know what would be.

Slavery was abolished in England in 1772, in Pennsylvania in 1780, in Canada in 1793 (Upper) and 1803 (Lower), and throughout the British Empire in 1833. The large, politically active advocacy groups against it in the English-speaking world were predominantly religious — Quakers and evangelicals — although secular philosophers such as Mill were opponents of slavery as well.

Slavery in Latin America started downhill when Chile declared no more children would be born slaves ("freedom of wombs") in 1811. Revolutionary France abolished slavery in its New World colonies in 1793-1795, after the Haitian slave uprising of 1791. However, slavery was reestablished by Napoleon; and had to be abolished again in 1848.

Notably, the "cold places abolished slavery first because they didn't have long growing seasons" idea falls flat — most of the tropical New World colonies had abolished slavery well before the U.S. did in 1865.

-1PhilGoetz12y
Some more competent historian would need to do that.

Take any moral advance you like. Study its history, and you'll find people adopted it when it became economically advantageous to those in power do so.

In other words, to falsify your entire theory, all I have to do is find an example where a moral choice came at considerable economic cost?

That doesn't sound too hard.

4juliawise12y
Prohibition in the US? I've heard arguments that it was all an effort to homogenize immigrants and the working class into some middle-class Protestant dream, but not that it was economically advantageous to the people in power.
0gwern12y
I had not heard that the Mafia lobbied hard for Prohibition, that is true. On the other hand, I had heard economic justifications: that working men were wasting their pay on booze in the corner saloon or bar, that alcohol damaged their health, etc. (On the gripping hand, when I was reading about the Prohibitionists, it definitely struck me at the time that these seemed like pretty hollow soldier-arguments which were definitely not the true reason people were teetotallers.)
7Vaniver12y
"If you look at the drug war from a purely economic point of view, the role of the government is to protect the drug cartel." -- Milton Friedman
4gwern12y
Which is mixing up the effect of the drug war with its purpose; the purpose is what matters here. (Why were the Prohibition activists trying to get Prohibition enacted, were their motives moral or economic? What actually happens is beside the point, and undermined by the eventual repeal of Prohibition besides.) Or are you suggesting that the small-time Mafia (or their more recent equivalents) foresaw the usefulness of Prohibition and the War on Drugs and directly contributed to their coming into existence eg. by bankrolling hardliner candidates? That would be remarkably insightful of them and one wonders how the mafia or cartels solved the public goods problem this represents...
0Vaniver12y
Corrupt city machines used control of liquor licenses and related bribes to great effect, and found prohibition and other hardline restrictions useful because they made discretion more valuable (and allowed harsher measures against opponents). For example, Roosevelt was engaged in a war against New York City's alcohol legislation as a police commissioner (because it was the largest cause of police corruption he was fighting) in 1895, about a generation before Prohibition passed nationwide.
2gwern12y
That's very interesting, but corrupt city machines don't gain revenue from liquor licenses if all licenses or consumption of any kind are banned, which leaves only bribery of police as a possible source; do you have any reason to believe the nation-wide movement to push through an entire Constitutional amendment, which succeeded in 46 of the 48 states, was even slightly assisted by the interest of would-be corrupt policemen? (Personally, if I were trying to explain Prohibition as a purely economic or rent-seeking phenomenon, consistent with OP, I'd be looking at anti-German and anti-beer sentiment rather than corrupt policemen...)
1dlthomas12y
This is not quite an accurate picture of prohibition in the US - there were religious exemptions, for instance, and I expect producers had to be licensed. I know there was a cap on the acreage of vineyards put towards sacramental wine (which seems a plausible feature of a regulatory apparatus that might involve licensing).
0Vaniver12y
Where do you think you buy employment as a policeman? I suspect it was mostly pushed by identity voters who didn't know how things would turn out, but I imagine that the eyes of clever gangsters and corrupt policemen all lit up when they heard about it. I suspect that they put little effort into opposing it, which could count as assistance. If any of them did support it, I imagine it was as secretly as they could manage, and thus it might be difficult for us to know about even now.
3Prismattic12y
Minor bit of historical non-trivia here: The Eighteenth Amendment was passed in 1920, before "One man, one vote." At the time, the US still had a "rotten borrough" problem, and furthermore, the average "wet" district had far more people in it than the average "dry" district. Prohibition passed in spite of the fact that a majority of voting-age citizens probably opposed it.
1TimS12y
You can falsify Marxist analysis that way, but not Foucault.
[-][anonymous]12y110

.

3TimS12y
A world with no moral change would falsify Foucault. ETA: And a world where all moral change was easily explained by some single variable theory (e.g. becoming more Christian or Marxist analysis) would also falsify Foucault.
0PhilGoetz12y
Are you just trying to score a rhetorical point by pointing out that I made a statement that wasn't meant to be literally true? Even if you believed that I meant it to be literally true - which I didn't - the reasonable, charitable response would be to say, "That's probably not literally true, but it doesn't matter as long as it's 80% true."
4gwern12y
Moral progress is slow. No one argues for a fast moral progress over the last 5000 years; this implies that moral progress has a hard time overcoming inertia - like economic factors. A low rate is not evidence for your theory, it is evidence for moral progress since it is exactly what one would expect of anything spread over 5000 years. So I charitably interpreted your theory as not being empirically inert and predicting an extremely low rate indeed, so low that a couple of examples would be enough to penalize it. You are free to withdraw that part or explain how it's actually a good thing that your theory and the moral progress theory predict the same thing re: people placing morals over profit.
7fubarobfusco12y
Well, except for Steven Pinker.
0gwern12y
I'm not sure he does either. He covers a very long sweep of time, and I don't think he points to any clear shifts until at least a millennium ago, although much of the changes comes in the past 500 years (which is also true of pretty much anything, that's why we call those shifts 'Revolutions').
1fubarobfusco12y
One of the first trends Pinker deals with is the transition from nonstate to state societies, including ancient empires. Hammurabi counts for some sort of moral progress over perpetually feuding hunter-gatherers.
3gwern12y
Does it? It is true Pinker spent a lot of time on trying to compare death-rates with hunter-gatherers, but it's not obvious that the comparison is that favorable for early empires (as opposed to modern civilizations) and I believe he also discusses ways in which people were worse off due to formation of states, such as poorer nutrition, taxation, and massively organized warfare. (It's a very big book and he covers a lot of nuances.)
1[anonymous]12y
.
0gwern12y
Yes.
2wedrifid12y
It is not at all certain that the change was an improvement in terms of moral behavior of the people in question.

Values converge and reach equilbrium in the same way that evolution converges and reaches equilbrium: Not at all.

Evolution constantly converges toward equilibrium. This is why when constructing evolutionary algorithms preventing premature convergence is such a big deal. In nature the main thing preventing convergence to a rather boring equilibrium is changing environmental conditions - followed by Fisherian runaway, of course.

0timtyler12y
Runaway forms of selection would seem to work against your point. Agents selecting for novelty would tend to disturb any evolutionary equilibrium.

I am confused.

You seem to be simultaneously arguing that 1) there's no objective way to define "better values", so we can't assert that our present values are any better than our past values, and 2) moral advances bring us farther from "equilibrium", so equilibrium isn't the right way to define better values.

But implicit in the second statement is the assumption that there are "better values", and "moral advances"!

Is the present more morally advanced than the past, or not? Were slavery and the end of masculinity (?... (read more)

1juliawise12y
I was confused about this, too. I would have understood better if there were either an explanation or a link to an explanation of "reflective equilibrium."
2KPier12y
It's a philosophical term, not a LessWrong one, and I've usually seen it defined as something like "examining moral judgments about a particular issue by looking for their coherence with our beliefs about similar cases and our beliefs about a broader range of moral and factual issues.". I'm not sure PhilGoetz is using it that way. seems hopelessly far from the standard understanding of reflective equilibrium, which requires "subjecting the views we encounter to extensive criticism from alternative moral perspectives". No one does that in the Iliad. But I am still confused about PhilGoetz's point, so I may be missing something.
1PhilGoetz12y
Why? How does being able to be farther from, or closer to, reflective equilibrium, assert something about the existence of better values? My argument about equilibrium is simply that moral changes over time are not biased to bring a moral system closer to equilibrium. It is a separate argument from whether or not those changes are moral improvements. Those two arguments are separate. (This is trivially true if you consider the starting point to be the null moral system with no morals. Organisms grow more complicated, and their morals grow more complicated with them. Extrapolating forward in time to superhumans is best imaginged by looking backwards in time to simpler organisms.)
5KPier12y
That's not my assertion; it's yours. "Consider an even more significant moral advance", you wrote in your section about masculinity. Are you being facetious, or do you believe that was a moral advance? If it was, how do we know? Could you explain what you mean by "reflective equilibrium", if it's not the standard definition? It seems to me that these two arguments are also different: Moral changes over time do not tend to bring a moral system closer to equilibrium. and Moral changes over time ought not bring a moral system closer to equilibrium. It seems to me that you are making a case for 1, but using it as an argument for 2. Am I still missing something?
[-]gjm12y40

Phil, in your opinion does the argument you offered for the thesis "There can be no evidence that morality has improved" likewise prove (mutatis mutandis) that there can be no evidence that knowledge can be improved? [EDIT, a few days later: of course I meant "has improved" at the end there. Sorry.]

It looks to me as if it should work as well for that as for your actual thesis: I cannot envisage a scenario in which we would believe that the opinions held by people in the past were better than our present opinions, and stably maintain tha... (read more)

2[anonymous]12y
Isn't this part of basis for many of the Abrahmic faiths, and possibly others? The revelation(s) was/were in the past, and the farther we get from those values, the worse off we get.
1gjm12y
That's a similar case to the example I gave, and it has the same features that (I think) make it unhelpful as a defence of Phil's argument: (1) in that situation, the past opinions that we think were better than our present ones are ones we don't know (and therefore can't choose to adopt), and (2) pretty much exactly the same scenario works pretty much exactly as well for values as for opinions.

If you have such a strong belief, that must mean you have evidence for it. That must mean you had some hypothesis, and the evidence could have gone either way; and the evidence went in such a way that it supported your hypothesis.

Not for particularly strong beliefs, but consider the case of judging certain pieces of art better (for your personal appreciation). What kind of evidence counts? Personal hunch seems to be the best we have.

When people talk about values getting better I think they mean that they have some core values and "improving" means they have a more accurate picture of what derivative values should be in order to maximize those core values.

0torekp12y
The way I would spin the same point is: when values improve in coherence, they improve in rationality, which can be assessed independently of whether or not one endorses the core values.

Morality has been getting better over time, right?

I don't understand the claim.

If you believe this, then in the comments below, please describe a scenario that could have happened, in which we would today believe that the values people had hundreds of years ago were superior to the values they have today.

It's not uncommon for societies to believe others more moral, particularly their ancestors, but not always. There is a whole noble savage genre. My first piece of evidence is that my society thinks it has improved on the past; I expect to find such... (read more)

I think there's good evidence against moral progress. Take any example somebody would give of moral progress and you can generally find another society or another era where the same, or at least similar, values were held. The appropriate question for somebody who believes in moral progress is, I think, What is the moral equivalent of a Saturn V rocket or a 747? Technological progress is obvious. I can point to any number of devices we have now that have absolutely no equivalent in history. Arguing that, say, animal welfare in the West is a genuine moral ad... (read more)

2fubarobfusco12y
Bear in mind that the mass adoption of technology often lags well behind the development of the scientific principles it uses. We're not all flying around in rockets; or even jet planes on a daily basis. Mightn't we expect something similar from moral progress? The fact that some idealist has proposed a moral principle isn't the same as it being generally adopted. The Saturn V rocket is from the '60s. What sort of moral progress in our society might have reached some sort of critical mass around then? Maybe something to do with this guy or even that guy too? The idea that members of social minorities should expect equal access to public goods, and equal protection against violence, was not a new idea in principle but it was a new implementation in practice (and one still being worked on).
-1timtyler12y
...right - but go back a bit further and lots of our ancestors were cannibals who bashed each other's skulls in and ate their brains in victory celebrations. Moral progress is pretty obvious too.

Is there any reason to think this process will converge, rather than diverge more and more, as it has for all of history? If there is, it has not been articulated.

Future creatures will probably have bigger genomes, bigger sef-descriptions, and so bigger moralities - assuming, of course, that their morality refers to themselves. There might be practical limits on creature size - but these are probably large, leaving a lot of space for evolution in the mean time.

The idea that values will freeze arises out of an analysis of self-improving systems, that ... (read more)

1PhilGoetz12y
How does a self-improving system improve itself, without discovering contradictions or gaps in its values? Does value freeze require knowledge freeze?
2timtyler12y
By getting a faster brain, more memory, more stored resources and a better world model, perhaps. Values don't have to have "contradictions" or "gaps" in them. Say you value printing out big prime numbers. Where are the contradictions or gaps going to come from? Usually values and knowledge are considered to be orthogoonal - so "no".

(I don't even know how to express coherently the idea that "values are getting better".)

Do you grant that I can have reflective preferences about the way my values should change in the future? That is, that I would not want my values to change in certain ways (e.g. by the intervention of an antagonist) but would want my values to change in other ways (e.g. if I think for a long time and decide that I value something different).

If so it seems clear that I can have preferences over ways my values could have changed in past, and can therefore sa... (read more)

4PhilGoetz12y
No. If you have a preference about how your values should change, it means you have conflicting values. If you think that you want your values to change, this probably means that the conscious you places more value on one preference, and the unconscious you places more value on another. This is what is happening when people say they wish they could eat less. Their minds want to eat less, and their bodies want to eat more.
0roystgnr12y
You and paulfchristiano seem to be using the word "way" in two different ways. Your post makes sense if I replace "the way" and "how" with "the direction in which". His makes sense if I replace "the way" by "the means with which". To apply your example: I can't consistently prefer a value change like "I should eat more fish", because if I wholeheartedly preferred that then I'd already be eating more fish. I can prefer a value change like "I should eat more of whatever foods are recommended by good nutritional studies that I haven't seen yet", because although I cannot identify any specific failing of my current values I can identify that there are specific ways in which they might be improved in the future by unexpected new information. This possibility of improvement applies only to instrumental values and self-inconsistent terminal values, but that's still pretty useful. How many people currently have and can unambiguously define self-consistent terminal values?
0TimS12y
Maybe you can have preferences about your future values, but most moral change is very slow. Do societies have coherent preferences about their future values? Before you say yes, consider the massive moral differences between us and some ancient ancestor society. Would Socrates really have predicted universal suffrage?
3PhilGoetz12y
Plato imagined women voting.
0fubarobfusco12y
Francis Godwin in the 1620s imagined traveling to the moon. Imagining progress is not the same as implementing it.

I think there are two steps to morality engineering, either of which can fail:

  1. Develop an ethical code through deliberate reflection, that is better than existing values.
  2. Bind that code into the active moral code.

You say neither has happened; I disagree on both, but I'll limit this post to the second question on "binding." I use the following definitions - they may not be correct or universal, but they should be internally consistent:

  • Value System: A collection of memes to do with decision-making, which provide better overall utility than in
... (read more)
0drc500free12y
Some Examples: Temple Judaism - Moral Development While emotionless ethical codes tend to be ineffective, morals can and have been engineered. This is done by careful manipulation of the binding layers. The "Ten Commandments" by itself is a prescriptive set of Ethics. The story of "Moses bringing the Ten Commandments" is a binding mechanism for that set of tenets, including an appeal to emotion (fear of God's wrath, as demonstrated in the story). Additional stories highlight each commandment, binding them with references to positive and negative emotions. The Torah as a revered source of stories packages these stories and adds another layer of binding with meta-stories about its own origin. The document is considered a holy and perfect source, with rituals for precisely copying, using, and destroying the physical scrolls. Several covenants between God and Man are detailed in the scrolls and provide emotionally-backed reciprocity. The stories are interwoven with sacrificial acts, including a ritualized and bloody sacrifice of the first born son with a lamb as a proxy, which hits primal communal and animal emotions. Outside of the text of the Torah, which contains stories and meta-stories, a set of rites and rituals related to the Torah itself increases emotional impact and exposure. This is a deliberate act of engineering by the Deuteronomist editors, who had to amalgate stories from multiple cultures (at minimum a nomadic, sheparding culture and an agrarian crop-based culture), to create the Temple religion. Rabbinical Judaism - Moral Engineering The destruction of the Second Temple was disasterous for the Temple-based moral system. Despite early use of writing and a fairly advanced scholarly culture, the reliance on a specific physical location had prevented any real territorial expansion. Significant parts of the moral code were supported by visceral sacrifice at the Temple for both internal consistency and emotional binding. Only two cultural branches seem

I think there are two steps to morality engineering, either of which can fail:

  1. Develop an ethical code through deliberate reflection, that is better than existing values.
  2. Bind that code into the active moral code.

You say neither has happened; I disagree on both, but I'll limit this post to the second question on "binding." I use the following definitions - they may not be correct or universal, but they should be internally consistent:

  • Value System: A collection of memes to do with decision-making, which provide better overall utility than in
... (read more)
[This comment is no longer endorsed by its author]Reply

CEV isn't just about your values changing by self-reflection. It's about your values changing as you become the person you wish you were. Humanity has reached the peak for self-reflection long ago (though I believe it still advances a little due to a growing understanding of the universe), but we have done nothing about human nature.

Your evidence against CEV is flawed, though you still correctly point out the lack of evidence for it. I doubt we would all end up in the same place, and it's likely that one person could end up in two very different places jus... (read more)

[-][anonymous]12y00

.

5PhilGoetz12y
I'm not convinced of that. The act of trying to converge requires learning and exploring ideas. This creates new concepts and new understandings, and situates morality in a higher-dimensional space with more known consequences to consider, and enables more-complex social structures. All these things (judging from history) make morality more complex faster than they iron out the inconsistencies. (I don't understand the last sentence - is Amanda Knox supposed to be particularly virtuous?)
-1[anonymous]12y
.
[-]TimS12y-40

As you've described it, Adaptiveness is very economic. But there are lots of social changes that are hard to explain economically. For example, the expansion of political inclusiveness in the West (Monarchy -> Limited Voting Rights -> Universal Manhood Suffrage -> Universal Suffrage).

And there is a plausible economic story for the creation of Jim Crow (rich whites trying to prevent poor whites from forming a political coalition with poor blacks). But I'm unpersuaded that there's a compelling economic explanation for the end of Jim Crow.

2Jayson_Virissimo12y
There are subfields within economics that attempt to explain precisely these kinds of "social change" using standard microeconomic theory (for instance, public choice). -Anthony de Jasay, The State
0TimS12y
Public choice is an excellent attack on the naive view that all politics is aimed at "improving society as a whole." And regulatory capture of agencies like the Civil Aeronautics Board, allowing airline rate setting that favored established airlines, is an expected outcome according to public choice theory. But the CAB was abolished eventually. More broadly, the expansion of so-called "minority rights" is not well-explained by economic theory. Even with the moral justifications supporting employment discrimination law, it is not accurate to say that prohibiting some reasons for hiring and firing workers is more efficient. At best, economic efficiency is unaffected.
1PhilGoetz12y
I don't know, but my first guess would be that these are cases where the oppressed gained enough power and ambition that it wasn't worthwhile to oppress them anymore. As a rule, giving some group a vote is easier than taking it away from them. That's a natural ratchet mechanism. You don't need to posit a historical trend towards better morals. There's always going to be one political party that would be better served at the moment by extending the vote to some group; and sometimes they'll have the power to do it.
0TimS12y
If there are one-way ratchets in the moral environment, why can't we define moral progress as going further into the one-way ratchets?
1TheOtherDave12y
We certainly can define "moral progress" that way, if we wish. It's just a phrase, we can define it any way we like. But we should take care, after so doing, not to assume that the properties we would naively associate with moral progress apply to the referent of "moral progress." In particular, a lot of people who talk about moral progress seem to believe it has something to do with people getting better over time by the speaker's standards. If any path involving one-way ratchets is by definition moral progress, then there exist paths of moral progress that involve people getting worse over time by most speakers' standards, so they ought to give up that belief if they're going to use that definition. It may be more productive to use a term less subject to misunderstanding.
0TimS12y
Yeah, my point is less interesting than I intended. Maybe a little to much "not my true rejection" on my part, since I think moral progress is a coherent (but false) assertion based on entirely different reasoning.
0PhilGoetz12y
We can say that universal suffrage is not moral progress if it usually leads to countries bankrupting themselves after people realize they can vote themselves money, and then being unable to solve the problem because moral posing wins more votes than reason and compromise.