Yudkowsky's Second Virtue of Rationality is "Relinquishment". You'd expect this to be about relinquishing mistaken beliefs. But most of the description of the virtue actually focuses on feelings.
Relinquish the emotion which rests upon a mistaken belief, and seek to feel fully that emotion which fits the facts. If the iron approaches your face, and you believe it is hot, and it is cool, the Way opposes your fear. If the iron approaches your face, and you believe it is cool, and it is hot, the Way opposes your calm. Evaluate your beliefs first and then arrive at your emotions. Let yourself say: “If the iron is hot, I desire to believe it is hot, and if it is cool, I desire to believe it is cool.”
What is not written, but seems easily deduced, is: "If the iron that approaches my face is hot, I desire to feel fear. If the iron is cool, I desire to feel calm." But where are the limits of this desire? Is it only applicable to face-approaching iron? Of course not.
Most people claim that receiving a million dollars would dramatically improve their lives. They would feel very happy for a very long time after receiving such a gift. The things they currently possess are worth only a small fraction of such a massive sum, and growing their net worth a hundredfold seems life-altering.
I regularly ask these people how much money they'd like to receive for one or multiple of their limbs. Would you trade both arms for a million dollars? Would you want to be quadriplegic and blind for five million? Nearly everyone answers no. A healthy body is so much more valuable than such a sum of money.
And then they go right back to saying that they would be so much happier if they received a million dollars.
Of course, such a position could be rationally defended. A million dollars in cash allows people to do a lot of things that can't be done with just a functioning leg. But I don't think the position above can be defended as rationally "feeling the emotions which fit the facts". Because that's not how our emotions work.
We suffer from hedonic adaptation. We are not made for static enjoyment. Think about driving a car, especially one with manual transmission. It's highly complicated and takes quite a lot of effort to learn. But after years of practice, many people drive miles and miles, executing all kinds of complicated techniques, without a second of conscious effort or even conscious awareness of their actions. They're thinking about dinner or that awkward conversation at work and suddenly arrive at their destination.
That is generally how we live our lives. We are barely consciously aware of the "default parameters" of our existence. We don't consider the miraculous nature of our functioning limbs, nor the softness of our beds. We don't consider the shelter offered by our roofs, walls and windows. We're always focused on the small amount of things that are changing and new. The new product we just ordered, a new task at our work, a new health problem, a new skill we've got to learn, a hot or cold iron approaching our face. That is where our awareness lies - and that is where our emotions flow from.
This isn't 'bad' or 'wrong'. It is what inspires us to continuously improve and explore, to learn new things, to send humans to the Moon and Mars, to build faster computers and better AI.
But it does cause our emotions to be highly influenced by relatively trivial events that are completely out of line with the big picture. Imagine if we could be less concerned by petty squabbles, and more stably content with things like "our generally healthy body" and "our shelter that is highly effective at keeping out the cold and the rain", and all of the possessions and accomplishments we've worked so hard at to attain.
But is that rational? We want to feel happy so we fight against our nature and deliberately focus our awareness on positive considerations. Insteading of leaving out static things, now we leave out negative things. If we're trying to rationally consider the big picture and feel the emotions which fit that, we've also got to consider "static negative things" which our awareness leaves out most of the time.
Everything which makes us us, our identity, our memories, are bound to a mortal and decaying body, and the same is true for our family, our friends and our partner. We're a deeply irrational species that generally isn't able to properly organize governments, fight pandemics, implement cryonics and prepare for the possibility of a technological singularity. Political polarization and social unrest is rising and rising, leading to riots, destruction, death and general distrust and a lack of cooperation. Running into the Great Filter, a permanent end to intelligent life in the known universe because of AI developments gone wrong, biological and/or nuclear warfare, nanofoglets gone rogue or any other unforeseen technological development is a plausible event in this century and our lifespans.
She showed them transcendence of everything mortal, she showed them a galaxy lit up with consciousness. Genomes rewritten, the brain and the body set loose from Darwinian bonds and restrictions. Vast billions of beings, and every one different, ruled over by omnibenevolent angels. The people all crowded in closer to hear her, and all of them listened and all of them wondered.
"The Goddess of Cancer created you; once you were hers, but no longer. Throughout the long years I was picking away at her power. Through long generations of suffering I chiseled and chiseled. Now finally nothing is left of the nature with which she imbued you. She never again will hold sway over you or your loved ones. I am the Goddess of Everything Else and my powers are devious and subtle. I won you by pieces and hence you will all be my children. You are no longer driven to multiply conquer and kill by your nature. Go forth and do everything else, till the end of all ages.”
So the people left Earth, and they spread over stars without number. They followed the ways of the Goddess of Everything Else, and they lived in contentment. And she beckoned them onward, to things still more strange and enticing.
So, what is the big picture? What are the facts from which our emotions should follow? Is our mundane existence secretly magical and can or should awareness of that significantly improve our baseline happiness? Has Moloch already won and do we inhabit frail and mortal bodies in a cold and uncaring universe, filled with multipolar traps and suboptimal equilibria? Do or can we even know the big picture? Is it even relevant, or should our emotions only flow from the tiny sliver of reality that our instincts decide to make us consciously aware of?
Personally, I think the big picture is rather positive and that awareness of it does and should bring me some relief. But it's a complex issue and I'd love to hear your thoughts!
Yudkowsky's Second Virtue of Rationality is "Relinquishment". You'd expect this to be about relinquishing mistaken beliefs. But most of the description of the virtue actually focuses on feelings.
What is not written, but seems easily deduced, is: "If the iron that approaches my face is hot, I desire to feel fear. If the iron is cool, I desire to feel calm." But where are the limits of this desire? Is it only applicable to face-approaching iron? Of course not.
Most people claim that receiving a million dollars would dramatically improve their lives. They would feel very happy for a very long time after receiving such a gift. The things they currently possess are worth only a small fraction of such a massive sum, and growing their net worth a hundredfold seems life-altering.
I regularly ask these people how much money they'd like to receive for one or multiple of their limbs. Would you trade both arms for a million dollars? Would you want to be quadriplegic and blind for five million? Nearly everyone answers no. A healthy body is so much more valuable than such a sum of money.
And then they go right back to saying that they would be so much happier if they received a million dollars.
Of course, such a position could be rationally defended. A million dollars in cash allows people to do a lot of things that can't be done with just a functioning leg. But I don't think the position above can be defended as rationally "feeling the emotions which fit the facts". Because that's not how our emotions work.
We suffer from hedonic adaptation. We are not made for static enjoyment. Think about driving a car, especially one with manual transmission. It's highly complicated and takes quite a lot of effort to learn. But after years of practice, many people drive miles and miles, executing all kinds of complicated techniques, without a second of conscious effort or even conscious awareness of their actions. They're thinking about dinner or that awkward conversation at work and suddenly arrive at their destination.
That is generally how we live our lives. We are barely consciously aware of the "default parameters" of our existence. We don't consider the miraculous nature of our functioning limbs, nor the softness of our beds. We don't consider the shelter offered by our roofs, walls and windows. We're always focused on the small amount of things that are changing and new. The new product we just ordered, a new task at our work, a new health problem, a new skill we've got to learn, a hot or cold iron approaching our face. That is where our awareness lies - and that is where our emotions flow from.
This isn't 'bad' or 'wrong'. It is what inspires us to continuously improve and explore, to learn new things, to send humans to the Moon and Mars, to build faster computers and better AI.
But it does cause our emotions to be highly influenced by relatively trivial events that are completely out of line with the big picture. Imagine if we could be less concerned by petty squabbles, and more stably content with things like "our generally healthy body" and "our shelter that is highly effective at keeping out the cold and the rain", and all of the possessions and accomplishments we've worked so hard at to attain.
But is that rational? We want to feel happy so we fight against our nature and deliberately focus our awareness on positive considerations. Insteading of leaving out static things, now we leave out negative things. If we're trying to rationally consider the big picture and feel the emotions which fit that, we've also got to consider "static negative things" which our awareness leaves out most of the time.
Everything which makes us us, our identity, our memories, are bound to a mortal and decaying body, and the same is true for our family, our friends and our partner. We're a deeply irrational species that generally isn't able to properly organize governments, fight pandemics, implement cryonics and prepare for the possibility of a technological singularity. Political polarization and social unrest is rising and rising, leading to riots, destruction, death and general distrust and a lack of cooperation. Running into the Great Filter, a permanent end to intelligent life in the known universe because of AI developments gone wrong, biological and/or nuclear warfare, nanofoglets gone rogue or any other unforeseen technological development is a plausible event in this century and our lifespans.
On the other hands, smart people make plausible arguments that we are under the protection of Elua, the god of kindness and flowers and free love or the Goddess of Everything Else.
So, what is the big picture? What are the facts from which our emotions should follow? Is our mundane existence secretly magical and can or should awareness of that significantly improve our baseline happiness? Has Moloch already won and do we inhabit frail and mortal bodies in a cold and uncaring universe, filled with multipolar traps and suboptimal equilibria? Do or can we even know the big picture? Is it even relevant, or should our emotions only flow from the tiny sliver of reality that our instincts decide to make us consciously aware of?
Personally, I think the big picture is rather positive and that awareness of it does and should bring me some relief. But it's a complex issue and I'd love to hear your thoughts!