= 27d567bc2d48d9f40e9ccc20ea370b15)
Pentashagon comments on An overall schema for the friendly AI problems: self-referential convergence criteria - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (110)
Even self-consistent morality is unstable if general self improvement allows for removal of values, even if removal is only a practical side effect of ignoring a value because it is more expensive to satisfy than other values. E.g. we (Westerners) generally no longer value honoring our ancestors (at least not many of them), even though it is a fairly independent value and roughly consistent with our other values. It is expensive to honor ancestors, and ancestors don't demand that we continue to maintain that value, so it receives less attention. We also put less value on the older definition of honor (as a thing to be defended and fought for and maintained at the expense of convenience) that earlier centuries had, despite its general consistency with other values for honesty, trustworthiness, social status, etc. I think this is probably for the same reason; it's expensive to maintain honor and most other values can be satisfied without it. In general, if U(moresatisfactionofvalue1) > U(moresatisfactionofvalue2) then maximization should tend to ignore value2 regardless of its consistency. If U(makevaluesselfconsistentvalue) > U(satisfyinganyother_value) then the obvious solution is to drop the other values and be done.
A sort of opposite approach is "make reality consistent with these pre-existing values" which involves finding a domain in reality state space under which existing values are self-consistent, and then trying to mold reality into that domain. The risk (unless you're a negative utilitarian) is that the domain is null. Finding the largest domain consistent with all values would make life more complex and interesting, so that would probably be a safe value. If domains form disjoint sets of reality with no continuous physical transitions between them then one would have to choose one physically continuous sub-domain and stick with it forever (or figure out how to switch the entire universe from one set to another). One could also start with preexisting values and compute a possible world where the values are self-consistent, then simulate it.
That's something different - a human trait that makes us want to avoid expensive commitments while paying them lip service. A self consistent system would not have this trait, and would keep "honor ancestors" in it, and do so or not depending on the cost and the interaction with other moral values.
If you want to look at even self-consistent systems being unstable, I suggest looking at social situations, where other entities reward value-change. Or a no-free-lunch result of the type "This powerful being will not trade with agents having value V."
This sweeps the model-dependence of "values" under the rug. The reason we don't value honoring our ancestors is that we don't believe they continue to exist after death, and so we don't believe social relations of any kind can be carried on with them.
This could be a case of typical mind fallacy. I can point to a number of statistical studies that show that a large number of Westerners claim that their ancestors do continue to exist after death.
Anyone who believes that some sort of heaven or hell exists.
And a lot of these people nonetheless don't accord their ancestors all that much in the way of honour...
They may believe it, but they don't alieve it.
How do you know?
Because the things that people would do if they believed in and acted as though they believe in life after death are profoundly weird, and we don't see any of that around. Can you imagine the same people who say that the dead "went to a better place" being sad that someone has not died, for instance? (Unless they're suffering so much or causing so much suffering that death is preferable even without an afterlife.)
I don't see why they need to be "profoundly weird". Remember, this subthread started with "honoring ancestors". The Chinese culture is probably the most obvious one where honoring ancestors is a big thing. What "profoundly weird" things does it involve?
Given that this is the Chinese we're talking about, expecting one's ancestors to improve investment returns in return for a good sacrifice.
Sorry, I don't know enough about Chinese culture to answer. But I'd guess that either they do have weird beliefs (that I'm not familiar with so I can't name them), or they don't and honoring ancestors is an isolated thing they do as a ritual. (The answer may be different for different people, of course.)
Speaking of "profoundly weird" things, does the veneration of saints in Catholicism qualify? :-)
Insofar as anyone expects saints to perform the function of demigods and intervene causally with miracles on behalf of the person praying, yes, it is "profoundly weird" magical thinking.
Why do you ask a site full of atheists if they think religion is irrational?
You are assuming that human beings are much more altruistic than they actually are. If your wife has the chance of leaving you and having a much better life where you will never hear from her again, you will not be sad if she does not take the chance.
Okay, now I'm curious. What exactly do you think that people would do if they believed in life after death?
-- Be happy that people have died and sad that they remain alive (same qualifiers as before: person is not suffering so much that even nothingness is preferable, etc.) and the reverse for people who they don't like
-- Want to kill people to benefit them (certainly, we could improve a lot of third world suffering by nuking places, if they have a bad life but a good afterlife. Note that the objection "their culture would die out" would not be true if there is an afterlife.)
-- In the case of people who oppose abortions because fetuses are people (which I expect overlaps highly with belief in life after death), be in favor of abortions if the fetus gets a good afterlife
-- Be less willing to kill their enemies the worse the enemy is
-- Do extensive scientific research trying to figure out what life after death is like.
-- Genuinely think that having their child die is no worse than having their child move away to a place where the child cannot contact them
-- Drastically reduce how bad they think death is when making public policy decisions; there would be still some effect because death is separation and things that cause death also cause suffering, but we act as though causing death makes some policy uniquely bad and preventing it uniquely good
-- Not oppose suicide
Edit: Support the death penalty as more humane than life imprisonment.
(Some of these might not apply if they believe in life after death but also in Hell, but that has its own bizarre consequences.)
Now might I do it pat. Now he is praying.
And now I’ll do ’t. And so he goes to heaven.
And so am I revenged.—That would be scanned.
A villain kills my father, and, for that,
I, his sole son, do this same villain send
To heaven.
Oh, this is hire and salary, not revenge.
He took my father grossly, full of bread,
With all his crimes broad blown, as flush as May.
And how his audit stands who knows save heaven?
But in our circumstance and course of thought
'Tis heavy with him. And am I then revenged
To take him in the purging of his soul
When he is fit and seasoned for his passage?
No.
Up, sword, and know thou a more horrid hent.
When he is drunk asleep, or in his rage,
Or in th' incestuous pleasure of his bed,
At game a-swearing, or about some act
That has no relish of salvation in ’t—
Then trip him, that his heels may kick at heaven,
And that his soul may be as damned and black
As hell, whereto it goes. My mother stays
This physic but prolongs thy sickly days.
-- Hamlet, Act 3, scene 3.
In Christianity, we are as soldiers on duty who cannot desert their post. Suicide and murder are mortal sins, damning one to perdition hereafter. Christians differ on whether this is a causal connection: works -> fate, or predestined by grace: grace -> works and grace -> fate. Either way, the consequences of believing in the Christian conception of life after death add up to practicing Christian virtue in this life.
In Buddhism, you get reincarnated, but only if you have lived a virtuous life do you get a favorable rebirth. Killing, including of yourself, is one of the worst sins and guarantees you a good many aeons in the hell worlds. The consequences of believing in the Buddhist conception of life after death add up to practicing Buddhist virtue in this life.
In Islam, paradise awaits the virtuous and hell the wicked. The consequences of believing in the Islamic conception of life after death add up to practicing Islamic virtue in this life. We can observe these consequences in current affairs.
I don't think that helps. For instance, if they alieve in an afterlife but their religion says that suicide and murder are mortal sins, they won't actually commit murder or suicide, but they would still not think it was sad that someone died in the way we think it's sad, would not insist that public policies should reduce deaths, etc.
You would also expect a lot of people to start thinking of religious prohibitions on murder and suicide like many people think of religious prohibitions on homosexuality--If God really wants that, he's being a jerk and hurting people for no obvious reason. And you'd expect believers to simply rationalize away religious prohibitions on murder and suicide and say that they don't apply just like religious believers already do to lots of other religious teachings (of which I'm sure you can name your own examples).
Hmmm.
What is known is that people who go to the afterlife don't generally come back (or, at least, don't generally come back with their memories intact). Historical evidence strongly suggests that anyone who remains alive will eventually die... so remaining alive means you have more time to enjoy what is nice here before moving on.
So, I don't imagine this would be the case unless the afterlife is strongly known to be significantly better than here.
Is it possible for people in the afterlife to have children? It may be that their culture will quickly run out of new members if they are all killed off. Again, though, this is only true if the afterlife is certain to be better than here.
Both true if and only if the afterlife is known to be better.
People have tried various experiments, like asking people who have undergone near-death experiences. However, there is very little data to work with and I know of no experiment that will actually give any sort of unambiguous result.
And where their child cannot contact anyone else who is still alive, either. Thrown into a strange and unfamiliar place with people who the parent knows nothing about. I can see that making parents nervous...
Exile is also generally considered uniquely bad; and since the dead have never been known to return, death is at the very least a form of exile that can never be revoked.
...depends. Many people who believe in life after death also believe that suicide makes things very difficult for the victim there.
Again, this depends; if there is a Hell, then the death penalty kills a person without allowing him much of a chance to try to repent, and could therefore be seen as less humane than life imprisonment.
The worse the afterlife is, the more similar people's reactions will be to a world where there is no afterlife. In the limit, the afterlife is as bad as or worse than nonexistence and people would be as death-averse as they are now. Except that this is contrary to how people claim to think of the afterlife when they assert belief in it. The afterlife can't be good enough to be comforting and still bad enough not to lead to any of the conclusions I described. And this includes being bad for reasons such as being like exile, being irreversible, etc.
And I already said that if there is a Hell (a selectively bad afterlife), many of these won't apply, but the existence of Hell has its own problems.
No, they believe-in-the-belief that their ancestors continue to exist after death. They rarely, and doubtingly, if ever, generate the concrete expectation that anything they can do puts them in causal contact with the ghosts of their ancestors, such that they would expect to see something different from their ancestors being permanently gone.