The lead article on everydayfeminism.com on March 25:

3 Reasons It’s Irrational to Demand ‘Rationalism’ in Social Justice Activism

The scenario is always the same: I say we should  abolish prisons,  police, and the  American settler state— someone tells me I’m irrational. I say we need  decolonization of the land — someone tells me I’m not being realistic.... When those who are the loudest, the most disruptive — the ones who want to destroy America and all of the oppression it has brought into the world — are being silenced even by others in social justice groups, that is unacceptable.

(The link from "decolonization" is to "Decolonization is not a metaphor", to make it clear s/he means actually giving the land back to the Native Americans.)

I regularly see people who describe how social justice activists act accused of setting up a straw man.  This article show that the bias of some SJWs against reason is impossible to strawman.  The author argues at length that rationality is bad, and that justice arguments shouldn't be rational or be defended rationally.  Ze is, or was, confused about what "rationality" means, but clearly now means it to include reason-based argumentation.

This isn't just some wacko's blog; it was chosen as the headline article for the website.  I had to click around to a few other articles to make sure it wasn't a parody site.

But it isn't just a sign of how irrational the social justice movement is—it has clues to how it got that way.

[Originally published at Intentional Insights in response to Religious and Rational]

Spirituality and rationality seem completely opposed. But are they really?

To get at this question, let's start with a little thought experiment.  Consider the following two questions:

1.  If you were given a choice between reading a physical book (or an e-book) or listening to an audiobook, which would you prefer?

2.  If you were given a choice between listening to music, or looking at the grooves of a phonograph record through a microscope, which would you prefer?

But I am more interested in the answer to a third question:

3.  For which of the first two questions do you have a stronger preference between the two options?

Most people will have a stronger preference in the second case than the first.  But why?  Both situations are in some sense the same: there is information being fed into your brain, in one case through your ears and in the other through your eyes.  So why should people's preference for ears be so much stronger in the case of music than books?

There is something in the essence of music that is lost in the translation between an audio and a visual rendering.  The same loss happens for words too, but to a much lesser extent.  Subtle shades of emphasis and tone of voice can convey essential information in spoken language. This is one of the reasons that email is so notorious for amplifying misunderstandings.  But the loss in much greater in the case of music.  

The same is true for other senses.  Color is one example.  A blind person can abstractly understand what light is, and that color is a byproduct of the wavelength of light, and that light is a form of electromagnetic radiation... yet there is no way for a blind person to experience subjectively the difference between red and blue and green.  But just because some people can't see colors doesn't mean that colors aren't real.

The same is true for spiritual experiences.

Now, before I expand that thought, I want to give you my bona fides.  I am a committed rationalist, and an atheist (though I don't like to self-identify as an atheist because I'd rather focus on what I *do* believe in rather than what I don't).  So I am not trying to convince you that God exists.  What I want to say is rather that certain kinds of spiritual experiences *might* be more than mere fantasies made up out of whole cloth. If we ignore this possibility we risk shutting ourselves off from a vital part of the human experience.

I grew up in the deep south (Kentucky and Tennessee) in a secular Jewish family.  When I was 12 my parents sent me to a Christian summer camp (there were no other kinds in Kentucky back in those days).  After a week of being relentlessly proselytized (read: teased and ostracized), I decided I was tired of being the camp punching bag and so I relented and gave my heart to Jesus.  I prayed, confessed my sins, and just like that I was a member of the club.

I experienced a euphoria that I cannot render into words, in exactly the same way that one cannot render into words the subjective experience of listening to music or seeing colors or eating chocolate or having sex.  If you have not experienced these things for yourself, no amount of description can fill the gap.  Of course, you can come to an *intellectual* understanding that "feeling the presence of the holy spirit" has nothing to do with any holy spirit. You can intellectually grasp that it is an internal mental process resulting from (probably) some kind of neurotransmitter released in response to social and internal mental stimulus.  But that won't allow you to understand *what it is like* any more than understanding physics will let you understand what colors look like or what music sounds like.

Happily, there are ways to stimulate the subjective experience that I'm describing other than accepting Jesus as your Lord and Savior. Meditation, for example, can produce similar results.  It can be a very powerful experience.  It can even become addictive, almost like a drug.

I am not necessarily advocating that you go try to get yourself a hit of religious euphoria (though I wouldn’t discourage you either -- the experience can give you some interesting and useful perspective on life).  Instead, I simply want to convince you to entertain the possibility that people might profess to believe in God for reasons other than indoctrination or stupidity.  Religious texts and rituals might be attempts to share real subjective experiences that, in the absence of a detailed modern understanding of neuroscience, can appear to originate from mysterious, subtle external sources.

The reason I want to convince you to entertain this notion is that an awful lot of energy gets wasted by arguing against religious beliefs on logical grounds, pointing out contradictions in the Bible and whatnot. Such arguments tend to be ineffective, which can be very frustrating for those who advance them. The antidote for this frustration is to realize that spirituality is not about logic.  It's about subjective experiences that not everyone is privy to.  Logic is about looking at the grooves.  Spirituality is about hearing the music.

The good news is that adopting science and reason doesn’t mean you have to give up on spirituality any more than you have to give up on music. There are myriad paths to spiritual experience, to a sense of awe and wonder at the grand tapestry of creation, to the essential existential mysteries of life and consciousness, to what religious people call “God.” Walking in the woods. Seeing the moons of Jupiter through a telescope. Gathering with friends to listen to music, or to sing, or simply to share the experience of being alive. Meditation. Any of these can be spiritual experiences if you allow them to be. In this sense, God is everywhere.

Once upon a time, in a lonely little village, beneath the boughs of a forest of burning trees, there lived a boy. The branches of the burning trees sometimes fell, and the magic in the wood permitted only girls to carry the fallen branches of the burning trees.

One day, a branch fell, and a boy was pinned beneath. The boy saw other boys pinned by branches, rescued by their girl friends, but he remained trapped beneath his own burning branch.

The fire crept closer, and the boy called out for help.

Finally, a friend of his own came, but she told him that she could not free him from the burning branch, because she already free'd her other friend from beneath a burning branch and he would be jealous if she did the same deed for anyone else. This friend left him where he lay, but she did promise to return and visit.

The fire crept closer, and the boy called out for help.

A man stopped, and gave the boy the advice that he'd get out from beneath the burning branch eventually if he just had faith in himself. The boy's reply was that he did have faith in himself, yet he remained trapped beneath the burning branch. The man suggested that perhaps he did not have enough faith, and left with nothing more to offer.

The fire crept closer, and the boy cried out for help.

A girl came along, and said she would free the boy from beneath the burning branch.

But no, her friends said, the boy was a stranger to her, was her heroic virtue worth nothing? Heroic deeds ought to be born from the heart, and made beautiful by love, they insisted. Simply hauling the branch off a boy she did not love would be monstrously crass, and they would not want to be friends with a girl so shamed.

So the girl changed her mind and left with her friends.

The fire crept closer. It began to lick at the boy's skin. A soothing warmth became an uncomfortable heat. The boy mustered his courage and chased the fear out of his own voice. He called out, but not for help. He called out for company.

A girl came along, and the boy asked if she would like to be friends. The girl's reply was that she would like to be friends, but that she spent most of her time on the other side of the village, so if they were to be friends, he must be free from beneath the burning branch.

The boy suggested that she free him from beneath the burning branch, so that they could be friends.

The girl replied that she once free'd a boy from beneath a burning branch who also promised to be her friend, but as soon as he was free he never spoke to her again. So how could she trust the boy's offer of friendship? He would say anything to be free.

The boy tried frantically to convince her that he was sincere, that he would be grateful and try with all his heart to be a good friend to the girl who free'd him, but she did not believe him and turned away from him and left him there to burn.

The fire crept closer and the boy whimpered in pain and fear as it spread from wood to flesh. He cried out for help. He begged for help. "Somebody, please!"

A man and a woman came along, and the man offered advice: he was once trapped beneath a burning branch for several years. The fire was magic, the pain was only an illusion. Perhaps it was sad that he was trapped but even so trapped the boy may lead a fulfilling life. Why, the man remembered etching pictures into his branch, befriending passers by, and making up songs.

The woman beside the man agreed, and told the boy that she hoped the right girl would come along and free him, but that he must not presume that he was entitled to any girl's heroic deed merely because he was trapped beneath a burning branch.

"But do I not deserve to be helped?" the boy pleaded, as the flames licked his skin.

"No, how wrong of you to even speak as though you do. My heroic deeds are mine to give, and to you I owe nothing," he was told.

"Perhaps I don't deserve help from you in particular, or from anyone in particular, but is it not so very cruel of you to say I do not deserve any help at all?" the boy pleaded. "Can a girl willing to free me from beneath this burning branch not be found and sent to my aide?"

"Of course not," he was told, "that is utterly unreasonable and you should be ashamed of yourself for asking. It is offensive that you believe such a girl may even exist. You've become burned and ugly, who would want to save you now?"

The fire spread, and the boy cried, screamed, and begged desperately for help from every passer by.

"It hurts it hurts it hurts oh why will no one free me from beneath this burning branch?!" he wailed in despair. "Anything, anyone, please! I don't care who frees me, I only wish for release from this torment!"

Many tried to ignore him, while others scoffed in disgust that he had so little regard for what a heroic deed ought to be. Some pitied him, and wanted to help, but could not bring themselves to bear the social cost, the loss of worth in their friends' and family's eyes, that would come of doing a heroic deed motivated, not by love, but by something lesser.

The boy burned, and wanted to die.

Another boy stepped forward. He went right up to the branch, and tried to lift it. The trapped boy gasped at the small relief from the burning agony, but it was only a small relief, for the burning branches could only be lifted by girls, and the other boy could not budge it. Though the effort was for naught, the first boy thanked him sincerely for trying.

The boy burned, and wanted to die. He asked to be killed.

He was told he had so much to live for, even if he must live beneath a burning branch. None were willing to end him, but perhaps they could do something else to make it easier for him to live beneath the burning branch? The boy could think of nothing. He was consumed by agony, and wanted only to end.

And then, one day, a party of strangers arrived in the village. Heroes from a village afar. Within an hour, one foreign girl came before the boy trapped beneath the burning branch and told him that she would free him if he gave her his largest nugget of gold.

Of course, the local villagers were shocked that this foreigner would sully a heroic deed by trafficking it for mere gold.

But, the boy was too desperate to be shocked, and agreed immediately. She free'd him from beneath the burning branch, and as the magical fire was drawn from him, he felt his burned flesh become restored and whole. He fell upon the foreign girl and thanked her and thanked her and thanked her, crying and crying tears of relief.

Later, he asked how. He asked why. The foreign girls explained that in their village, heroic virtue was measured by how much joy a hero brought, and not by how much she loved the ones she saved.

The locals did not like the implication that their own way might not have been the best way, and complained to the chief of their village. The chief cared only about staying in the good graces of the heroes of his village, and so he outlawed the trading of heroic deeds for other commodities.

The foreign girls were chased out of the village.

And then a local girl spoke up, and spoke loud, to sway her fellow villagers. The boy recognized her. It was his friend. The one who had promised to visit so long ago.

But she shamed the boy, for doing something so crass as trading gold for a heroic deed. She told him he should have waited for a local girl to free him from beneath the burning branch, or else grown old and died beneath it.

To garner sympathy from her audience, she sorrowfully admitted that she was a bad friend for letting the boy be tempted into something so disgusting. She felt responsible, she claimed, and so she would fix her mistake.

The girl picked up a burning branch. Seeing what she was about to do, the boy begged and pleaded for her to reconsider, but she dropped the burning branch upon the boy, trapping him once more.

The boy screamed and begged for help, but the girl told him that he was morally obligated to learn to live with the agony, and never again voice a complaint, never again ask to be free'd from beneath the burning branch.

"Banish me from the village, send me away into the cold darkness, please! Anything but this again!" the boy pleaded.

"No," he was told by his former friend, "you are better off where you are, where all is proper."

In the last extreme, the boy made a grab for his former friend's leg, hoping to drag her beneath the burning branch and free himself that way, but she evaded him. In retaliation for the attempt to defy her, she had a wall built around the boy, so that none would be able, even if one should want to free him from beneath the burning branch.

With all hope gone, the boy broke and became numb to all possible joys. And thus, he died, unmourned.

If you are a person who finds it difficult to tell "no" to their friends, this one weird trick may save you a lot of time!

Scenario 1

Alice: "Hi Bob! You are a programmer, right?"

Bob: "Hi Alice! Yes, I am."

Alice: "I have this cool idea, but I need someone to help me. I am not good with computers, and I need someone smart whom I could trust, so they wouldn't steal my idea. Would you have a moment to listen to me?"

Alice explains to Bob her idea that would completely change the world. Well, at the least the world of bicycle shopping.

Instead of having many shops for bicycles, there could be one huge e-shop that would collect all the information about bicycles from all the existing shops. The customers would specify what kind of a bike they want (and where they live), and the system would find all bikes that fit the specification, and display them ordered by lowest price, including the price of delivery; then it would redirect them to the specific page of the specific vendor. Customers would love to use this one website, instead of having to visit multiple shops and compare. And the vendors would have to use this shop, because that's where the customers would be. Taking a fraction of a percent from the sales could make Alice (and also Bob, if he helps her) incredibly rich.

Bob is skeptical about it. The project suffers from the obvious chicken-and-egg problem: without vendors already there, the customers will not come (and if they come by accident, they will quickly leave, never to return again); and without customers already there, there is no reason for the vendors to cooperate. There are a few ways how to approach this problem, but the fact that Alice didn't even think about it is a red flag. She also has no idea who are the big players in the world of bicycle selling; and generally she didn't do her homework. But after pointing out all these objections, Alice still remains super enthusiastic about the project. She promises she will take care about everything -- she just cannot write code, and she needs Bob's help for this part.

Bob believes strongly in the division of labor, and that friends should help each other. He considers Alice his friend, and he will likely need some help from her in the future. Fact is, with perfect specification, he could make the webpage in a week or two. But he considers bicycles to be an extremely boring topic, so he wants to spend as little time as possible on this project. Finally, he has an idea:

"Okay, Alice, I will make the website for you. But first I need to know exactly how the page will look like, so that I don't have to keep changing it over and over again. So here is the homework for you -- take a pen and paper, and make a sketch of how exactly the web will look like. All the dialogs, all the buttons. Don't forget logging in and logging out, editing the customer profile, and everything else that is necessary for the website to work as intended. Just look at the papers and imagine that you are the customer: where exactly would you click to register, and to find the bicycle you want? Same for the vendor. And possibly a site administrator. Also give me the list of criteria people will use to find the bike they want. Size, weight, color, radius of wheels, what else? And when you have it all ready, I will make the first version of the website. But until then, I am not writing any code."

Alice leaves, satisfied with the outcome.

This happened a year ago.

No, Alice doesn't have the design ready, yet. Once in a while, when she meets Bob, she smiles at him and apologizes that she didn't have the time to start working on the design. Bob smiles back and says it's okay, he'll wait. Then they change the topic.

Scenario 2

Cyril: "Hi Diana! You speak Spanish, right?"

Diana: "Hi Cyril! Yes, I do."

Cyril: "You know, I think Spanish is the most cool language ever, and I would really love to learn it! Could you please give me some Spanish lessons, once in a while? I totally want to become fluent in Spanish, so I could travel to Spanish-speaking countries and experience their culture and food. Would you please help me?"

Diana is happy that someone takes interest in her favorite hobby. It would be nice to have someone around she could practice Spanish conversation with. The first instinct is to say yes.

But then she remembers (she knows Cyril for some time; they have a lot of friends in common, so they meet quite regularly) that Cyril is always super enthusiastic about something he is totally going to do... but when she meets him next time, he is super enthusiastic about something completely different; and she never heard about him doing anything serious about his previous dreams.

Also, Cyril seems to seriously underestimate how much time does it take to learn a foreign language fluently. Some lessons, once in a while will not do it. He also needs to study on his own. Preferably every day, but twice a week is probably a minimum, if he hopes to speak the language fluently within a year. Diana would be happy to teach someone Spanish, but not if her effort will most likely be wasted.

Diana: "Cyril, there is this great website called Duolingo, where you can learn Spanish online completely free. If you give it about ten minutes every day, maybe after a few months you will be able to speak fluently. And anytime we meet, we can practice the vocabulary you have already learned."

This would be the best option for Diana. No work, and another opportunity to practice. But Cyril insists:

"It's not the same without the live teacher. When I read something from the textbook, I cannot ask additional questions. The words that are taught are often unrelated to the topics I am interested in. I am afraid I will just get stuck with the... whatever was the website that you mentioned."

For Diana this feels like a red flag. Sure, textbooks are not optimal. They contain many words that the student will not use frequently, and will soon forget them. On the other hand, the grammar is always useful; and Diana doesn't want to waste her time explaining the basic grammar that any textbook could explain instead. If Cyril learns the grammar and some basic vocabulary, then she can teach him all the specialized vocabulary he is interested in. But now it feels like Cyril wants to avoid all work. She has to draw a line:

"Cyril, this is the address of the website." She takes his notebook and writes 'www.duolingo.com'. "You register there, choose Spanish, and click on the first lesson. It is interactive, and it will not take you more than ten minutes. If you get stuck there, write here what exactly it was that you didn't understand; I will explain it when we meet. If there is no problem, continue with the second lesson, and so on. When we meet next time, tell me which lessons you have completed, and we will talk about them. Okay?"

Cyril nods reluctantly.

This happened a year ago.

Cyril and Diana have met repeatedly during the year, but Cyril never brought up the topic of Spanish language again.

Scenario 3

Erika: "Filip, would you give me a massage?"

Filip: "Yeah, sure. The lotion is in the next room; bring it to me!"

Erika brings the massage lotion and lies on the bed. Filip massages her back. Then they make out and have sex.

This happened a year ago. Erika and Filip are still a happy couple.

Filip's previous relationships didn't work well, in long term. In retrospect, they all followed a similar scenario. At the beginning, everything seemed great. Then at some moment the girl started acting... unreasonably?... asking Filip to do various things for her, and then acting annoyed when Filip did exactly what he was asked to do. This happened more and more frequently, and at some moment she broke up with him. Sometimes she provided explanation for breaking up that Filip was unable to decipher.

Filip has a friend who is a successful salesman. Successful both professionally and with women. When Filip admitted to himself that he is unable to solve the problem on his own, he asked his friend for advice.

"It's because you're a f***ing doormat," said the friend. "The moment a woman asks you to do anything, you immediately jump and do it, like a well-trained puppy. Puppies are cute, but not attractive. Have you ready any of those books I sent you, like, ten years ago? I bet you didn't. Well, it's all there."

Filip sighed: "Look, I'm not trying to become a pick-up artist. Or a salesman. Or anything. No offense, but I'm not like you, personality-wise, I never have been, and I don't want to become your - or anyone else's - copy. Even if it would mean greater success in anything. I prefer to treat other people just like I would want them to treat me. Most people reciprocate nice behavior; and those who don't, well, I avoid them as much as possible. This works well with my friends. It also works with the girls... at the beginning... but then somehow... uhm... Anyway, all your books are about manipulating people, which is ethically unacceptable for me. Isn't there some other way?"

"All human interaction is manipulation; the choice is between doing it right or wrong, acting consciously or driven by your old habits..." started the friend, but then he gave up. "Okay, I see you're not interested. Just let me show you the most obvious mistake you make. You believe that when you are nice to people, they will perceive you as nice, and most of them will reciprocate. And when you act like an asshole, it's the other way round. That's correct, on some level; and in a perfect world this would be the whole truth. But on a different level, people also perceive nice behavior as weakness; especially if you do it habitually, as if you don't have any other option. And being an asshole obviously signals strength: you are not afraid to make other people angry. Also, in long term, people become used to your behavior, good or bad. The nice people don't seem so nice anymore, but they still seem weak. Then, ironicaly, if the person well-known to be nice refuses to do something once, people become really angry, because their expectations were violated. And if the asshole decides to do something nice once, they will praise him, because he surprised them pleasantly. You should be an asshole once in a while, to make people see that you have a choice, so they won't take your niceness for granted. Or if your girlfriend wants something from you, sometimes just say no, even if you could have done it. She will respect you more, and then she will enjoy more the things you do for her."

Filip: "Well, I... probably couldn't do that. I mean, what you say seems to make sense, however much I hate to admit it. But I can't imagine doing it myself, especially to a person I love. It's just... uhm... wrong."

"Then, I guess, the very least you could do is to ask her to do something for you first. Even if it's symbolic, that doesn't matter; human relationships are mostly about role-playing anyway. Don't jump immediately when you are told to; always make her jump first, if only a little. That will demonstrate strength without hurting anyone. Could you do that?"

Filip wasn't sure, but at the next opportunity he tried it, and it worked. And it kept working. Maybe it was all just a coincidence, maybe it was a placebo effect, but Filip doesn't mind. At first it felt kinda artificial, but then it became natural. And later, to his surprise, Filip realized that practicing these symbolic demands actually makes it easier to ask when he really needed something. (In which case sometimes he was asked to do something first, because his girlfriend -- knowingly or not? he never had the courage to ask -- copied the pattern; or maybe she has already known it long before. But he didn't mind that either.)

The lesson is: If you find yourself repeatedly in situations where people ask you to do something for them, but at the end they don't seem to appreciate what you did for them, or don't even care about the thing they asked you to do... and yet you find it difficult to say "no"... ask them to contribute to the project first.

This will help you get rid of the projects they don't care about (including the ones they think they care about in far mode, but do not care about enough to actually work on them in near mode) without being the one who refuses cooperation. Also, the act of asking the other person to contribute, after being asked to do something for them, mitigates the status loss inherent in working for them.

Alice: "I just flipped a coin [large number] times. Here's the sequence I got:

(Alice presents her sequence.)

Bob: No, you didn't. The probability of having gotten that particular sequence is 1/2^[large number]. Which is basically impossible. I don't believe you.

Alice: But I had to get some sequence or other. You'd make the same claim regardless of what sequence I showed you.

Bob: True. But am I really supposed to believe you that a 1/2^[large number] event happened, just because you tell me it did, or because you showed me a video of it happening, or even if I watched it happen with my own eyes? My observations are always fallible, and if you make an event improbable enough, why shouldn't I be skeptical even if I think I observed it?

Alice: Someone usually wins the lottery. Should the person who finds out that their ticket had the winning numbers believe the opposite, because winning is so improbable?

Bob: What's the difference between finding out you've won the lottery and finding out that your neighbor is a 500 year old vampire, or that your house is haunted by real ghosts? All of these events are extremely improbable given what we know of the world.

Alice: There's improbable, and then there's impossible. 500 year old vampires and ghosts don't exist.

Bob: As far as you know. And I bet more people claim to have seen ghosts than have won more than 100 million dollars in the lottery.

Alice: I still think there's something wrong with your reasoning here.

In risk modeling, there is a well-known distinction between aleatory and epistemic uncertainty, which is sometimes referred to, or thought of, as irreducible versus reducible uncertainty. Epistemic uncertainty exists in our map; as Eliezer put it, “The Bayesian says, ‘Uncertainty exists in the map, not in the territory.’” Aleatory uncertainty, however, exists in the territory. (Well, at least according to our map that uses quantum mechanics, according to Bells Theorem – like, say, the time at which a radioactive atom decays.) This is what people call quantum uncertainty, indeterminism, true randomness, or recently (and somewhat confusingly to myself) ontological randomness – referring to the fact that our ontology allows randomness, not that the ontology itself is in any way random. It may be better, in Lesswrong terms, to think of uncertainty versus randomness – while being aware that the wider world refers to both as uncertainty. But does the distinction matter?

To clarify a key point, many facts are treated as random, such as dice rolls, are actually mostly uncertain – in that with enough physics modeling and inputs, we could predict them. On the other hand, in chaotic systems, there is the possibility that the “true” quantum randomness can propagate upwards into macro-level uncertainty. For example, a sphere of highly refined and shaped uranium that is *exactly* at the critical mass will set off a nuclear chain reaction, or not, based on the quantum physics of whether the neutrons from one of the first set of decays sets off a chain reaction – after enough of them decay, it will be reduced beyond the critical mass, and become increasingly unlikely to set off a nuclear chain reaction. Of course, the question of whether the nuclear sphere is above or below the critical mass (given its geometry, etc.) can be a difficult to measure uncertainty, but it’s not aleatory – though some part of the question of whether it kills the guy trying to measure whether it’s just above or just below the critical mass will be random – so maybe it’s not worth finding out. And that brings me to the key point.

In a large class of risk problems, there are factors treated as aleatory – but they may be epistemic, just at a level where finding the “true” factors and outcomes is prohibitively expensive. Potentially, the timing of an earthquake that would happen at some point in the future could be determined exactly via a simulation of the relevant data. Why is it considered aleatory by most risk analysts? Well, doing it might require a destructive, currently technologically impossible deconstruction of the entire earth – making the earthquake irrelevant. We would start with measurement of the position, density, and stress of each relatively macroscopic structure, and the perform a very large physics simulation of the earth as it had existed beforehand. (We have lots of silicon from deconstructing the earth, so I’ll just assume we can now build a big enough computer to simulate this.) Of course, this is not worthwhile – but doing so would potentially show that the actual aleatory uncertainty involved is negligible. Or it could show that we need to model the macroscopically chaotic system to such a high fidelity that microscopic, fundamentally indeterminate factors actually matter – and it was truly aleatory uncertainty. (So we have epistemic uncertainty about whether it’s aleatory; if our map was of high enough fidelity, and was computable, we would know.)

It turns out that most of the time, for the types of problems being discussed, this distinction is irrelevant. If we know that the value of information to determine whether something is aleatory or epistemic is negative, we can treat the uncertainty as randomness. (And usually, we can figure this out via a quick order of magnitude calculation; Value of Perfect information is estimated to be worth $100 to figure out which side the dice lands on in this game, and building and testing / validating any model for predicting it would take me at least 10 hours, my time is worth at least $25/hour, it’s negative.) But sometimes, slightly improved models, and slightly better data, are feasible – and then worth checking whether there is some epistemic uncertainty that we can pay to reduce. In fact, for earthquakes, we’re doing that – we have monitoring systems that can give several minutes of warning, and geological models that can predict to some degree of accuracy the relative likelihood of different sized quakes.

So, in conclusion; most uncertainty is lack of resolution in our map, which we can call epistemic uncertainty. This is true even if lots of people call it “truly random” or irreducibly uncertain – or if they are fancy, aleatory uncertainty. Some of what we assume is uncertainty is really randomness. But lots of the epistemic uncertainty can be safely treated as aleatory randomness, and value of information is what actually makes a difference. And knowing the terminology used elsewhere can be helpful.

One of the enduring traits that I see in most characterizations of artificial intelligences is the idea that an AI would have all of the skills that computers have. It's often taken for granted that a general artificial intelligence would be able to perfectly recall information, instantly multiply and divide 5 digit numbers, and handily defeat Gary Kasparov at chess. For whatever reason, the capabilities of a digital intelligence are always seen as encompassing the entire current skill set of digital machines.

But this belief is profoundly strange. Consider how much humans struggle to learn arithmetic. Basic arithmetic is really simple. You can build a bare bones electronic calculator/arithmetic logic unit on a breadboard in a weekend. Yet humans commonly spend years learning how to perform those same simple operations. And the mental arithmetic equipment humans assemble at the end of this is still relatively terrible: slow, labor intensive, and prone to frequent mistakes. 

It is not totally clear why humans are this bad at math. It is almost certainly unrelated to brains computing using neurons instead of transistors. Based on personal experience and a cursory literature review, counting seems to rely primarily on identifying repeated structures in a linked list, and seems to be stored as verbal memory. When we first learn the most basic arithmetic we rely on visual pattern matching, and as we do more math basic math operations get stored in a look-up table in verbal memory. This is an absolutely bonkers way to implement arithmetic. 

While humans may be generally intelligent, that general intelligence seems to be accomplished using some fairly inelegant kludges. We seem to have a preferred framework for understanding built on our visual and verbal systems, and we tend to shoehorn everything else into that framework. But there's nothing uniquely human about that problem. It seems to be characteristic of learning algorithms in general, and so if our artificial learner started off by learning skills unrelated to math, it might learn arithmetic via a similarly convoluted process. While current digital machines do arithmetic via a very efficient process, a digital mind that has to learn those patterns may arrive at a solution as slow and convoluted as the one humans rely on.

In order to ensure that this post delivers what it promises, I have added the following content warnings:

Content Notes:

Pure Hypothetical Situation: The claim that perfect theoretical rationality doesn't exist is restricted to a purely hypothetical situation. No claim is being made that this applies to the real world. If you are only interested in how things apply to the real world, then you may be disappointed to find out that this is an exercise left to the reader.

Technicality Only Post: This post argues that perfectly theoretical rationality doesn't exist due to a technicality. If you were hoping for this post to deliver more, well, you'll probably be disappointed.

Contentious Definition: This post (roughly) defines perfect rationality as the ability to maximise utility. This is based on Wikipedia, which defines rational agents as an agent that: "always chooses to perform the action with the optimal expected outcome for itself from among all feasible actions". 

We will define the number choosing game as follows. You name any single finite number x. You then gain x utility and the game then ends. You can only name a finite number, naming infinity is not allowed.

Clearly, the agent that names x+1 is more rational than the agent that names x (and behaves the same in every other situation). However, there does not exist a completely rational agent, because there does not exist a number that is higher than every other number. Instead, the agent who picks 1 is less rational than the agent who picks 2 who is less rational than the agent who picks 3 and so on until infinity. There exists an infinite series of increasingly rational agents, but no agent who is perfectly rational within this scenario.

Furthermore, this hypothetical doesn't take place in our universe, but in a hypothetical universe where we are all celestial beings with the ability to choose any number however large without any additional time or effort no matter how long it would take a human to say that number. Since this statement doesn't appear to have been clear enough (judging from the comments), we are explicitly considering a theoretical scenario and no claims are being made about how this might or might not carry over to the real world. In other words, I am claiming the the existence of perfect rationality does not follow purely from the laws of logic. If you are going to be difficult and argue that this isn't possible and that even hypothetical beings can only communicate a finite amount of information, we can imagine that there is a device that provides you with utility the longer that you speak and that the utility it provides you is exactly equal to the utility you lose by having to go to the effort to speak, so that overall you are indifferent to the required speaking time.

In the comments, MattG suggested that the issue was that this problem assumed unbounded utility. That's not quite the problem. Instead, we can imagine that you can name any number less than 100, but not 100 itself. Further, as above, saying a long number either doesn't cost you utility or you are compensated for it. Regardless of whether you name 99 or 99.9 or 99.9999999, you are still choosing a suboptimal decision. But if you never stop speaking, you don't receive any utility at all.

I'll admit that in our universe there is a perfectly rational option which balances speaking time against the utility you gain given that we only have a finite lifetime and that you want to try to avoid dying in the middle of speaking the number which would result in no utility gained. However, it is still notable that a perfectly rational being cannot exist within a hypothetical universe. How exactly this result applies to our universe isn't exactly clear, but that's the challenge I'll set for the comments. Are there any realistic scenarios where the lack of existence of perfect rationality has important practical applications?

Furthermore, there isn't an objective line between rational and irrational. You or I might consider someone who chose the number 2 to be stupid. Why not at least go for a million or a billion? But, such a person could have easily gained a billion, billion, billion utility. No matter how high a number they choose, they could have always gained much, much more without any difference in effort.

I'll finish by providing some examples of other games. I'll call the first game the Exploding Exponential Coin Game. We can imagine a game where you can choose to flip a coin any number of times. Initially you have 100 utility. Every time it comes up heads, your utility triples, but if it comes up tails, you lose all your utility. Furthermore, let's assume that this agent isn't going to raise the Pascal's Mugging objection. We can see that the agent's expected utility will increase the more times they flip the coin, but if they commit to flipping it unlimited times, they can't possibly gain any utility. Just as before, they have to pick a finite number of times to flip the coin, but again there is no objective justification for stopping at any particular point.

Another example, I'll call the Unlimited Swap game. At the start, one agent has an item worth 1 utility and another has an item worth 2 utility. At each step, the agent with the item worth 1 utility can choose to accept the situation and end the game or can swap items with the other player. If they choose to swap, then the player who now has the 1 utility item has an opportunity to make the same choice. In this game, waiting forever is actually an option. If your opponents all have finite patience, then this is the best option. However, there is a chance that your opponent has infinite patience too. In this case you'll both miss out on the 1 utility as you will wait forever. I suspect that an agent could do well by having a chance of waiting forever, but also a chance of stopping after a high finite number. Increasing this finite number will always make you do better, but again, there is no maximum waiting time.

(This seems like such an obvious result, I imagine that there's extensive discussion of it within the game theory literature somewhere. If anyone has a good paper that would be appreciated).

Link to part 2: Consequences of the Non-Existence of Rationality 

I grew up in an atheistic household.

Almost needless to say, I was relatively hostile towards religion for most of my early life.  A few things changed that.

First, the apology of a pastor.  A friend of mine was proselytizing at me, and apparently discussed it with his pastor; the pastor apologized to my parents, and explained to my friend he shouldn't be trying to convert people.  My friend apologized to me after considering the matter.  We stayed friends for a little while afterwards, although I left that school, and we lost contact.

I think that was around the time that I realized that religion is, in addition to being a belief system, a way of life, and not necessarily a bad one.

The next was actually South Park's Mormonism episode, which pointed out that a belief system could be desirable on the merits of the way of life it represented, even if the beliefs themselves are stupid.  This tied into Douglas Adam's comment on Feng Shui, that "...if you disregard for a moment the explanation that's actually offered for it, it may be there is something interesting going on" - which is to say, the explanation for the belief is not necessarily the -reason- for the belief, and that stupid beliefs may actually have something useful to offer - which then requires us to ask whether the beliefs are, in fact, stupid.

Which is to say, beliefs may be epistemically irrational while being instrumentally rational.

The next peace I made with belief actually came from quantum physics, and reading about how there were several disparate and apparently contradictory mathematical systems, which all predicted the same thing.  It later transpired that they could all be generalized into the same mathematical system, but I hadn't read that far before the isomorphic nature of truth occurred to me; you can have multiple contradictory interpretations of the same evidence that all predict the same thing.

Up to this point, however, I still regarded beliefs as irrational, at least on an epistemological basis.

The next peace came from experiences living in a house that would have convinced most people that ghosts are real, which I have previously written about here.  I think there are probably good explanations for every individual experience even if I don't know them, but am still somewhat flummoxed by the fact that almost all the bizarre experiences of my life all revolve around the same physical location.  I don't know if I would accept money to live in that house again, which I guess means that I wouldn't put money on the bet that there wasn't something fundamentally odd about the house itself - a quality of the house which I think the term "haunted" accurately conveys, even if its implications are incorrect.

If an AI in a first person shooter dies every time it walks into a green room, and experiences great disutility for death, how many times must it walk into a green room before it decides not to do that anymore?  I'm reasonably confident on a rational level that there was nothing inherently unnatural about that house, nothing beyond explanation, but I still won't "walk into the green room."

That was the point at which I concluded that beliefs can be -rational-.  Disregard for a moment the explanation that's actually offered for them, and just accept the notion that there may be something interesting going on underneath the surface.

If we were to hold scientific beliefs to the same standard we hold religious beliefs - holding the explanation responsible rather than the predictions - scientific beliefs really don't come off looking that good.  The sun isn't the center of the universe; some have called this theory "less wrong" than an earth-centric model of the universe, but that's because the -predictions- are better; the explanation itself is still completely, 100% wrong.

Likewise, if we hold religious beliefs to the same standard we hold scientific beliefs - holding the predictions responsible rather than the explanations - religious beliefs might just come off better than we'd expect.