Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.
"Reality is normal." That is: Surprise, confusion, and mystery are features of maps, not of territories. If you would think like reality, cultivate outrage at yourself for failing to intuit the data, not resentment at the data for being counter-intuitive.
"Not one unusual thing has ever happened." That is: Ours is a tight-knit and monochrome country. The cosmos is simple, tidy, lawful. "[T]here is no surprise from a causal viewpoint — no disruption of the physical order of the universe."
"It all adds up to normality." That is: Whatever is true of fundamental reality does not exist in a separate universe from our everyday activities. It composes those activities. The perfected description of our universe must in principle allow us to reproduce the appearances we started with.
These maxims are remedies to magical mereology, anthropocentrism, and all manner of philosophical panic. But reading too much (or too little) into them can lead seekers from the Path. For instance, they may be wrongly taken to mean that the world is obliged to validate our initial impressions or our untrained intuitions. As a further corrective, I suggest: Reality is weirdly normal. It's "normal" in odd ways, by strange means, in surprising senses.
At the risk of vivisecting poetry, and maybe of stating the obvious, I'll point out that the maxims mean different things by "normal". In the first two, what's "normal" or "usual" is the universe taken on its own terms — the cosmos as it sees itself, or as an ideally calibrated demon would see it. In the third maxim, what's "normal" is the universe humanity perceives — though this still doesn't identify normality with what's believed or expected. Actually, it will take some philosophical work to articulate just what Egan's "normality" should amount to. I'll start with Copernicanism and reductionism, and then I'll revisit that question.
Everything is usual. Very nearly nothing is familiar.
- Spell 1: Since the beginning, not one unusual thing has ever happened. And intelligence explosion sounds... well, 'unusual' is the mildest word that comes to mind. So it won't happen.
- Counterspell: Reality is weirdly normal. It's not the kind of normal that sounds normal.
Relative to the universe's laws, everything is par for the course. But relative to human standards of normality — what I'll call the familiar — barely anything that actually exists is par for the course. Nearly all events are senseless, bizarre, inhuman. But that's because our mapping hardware is adapted to a very specific environment; if anything's objectively weird, it's we, not the other denizens of Everythingland. That's much of why absurdity is a poor guide to probability.
Egan's Law reminds us that beneath our human surface weirdness lies a deeper regularity, a deeper unity with the rest of Nature. But to call this alien order 'regular' already assumes a shift in perspective from what a person off the street would initially think of as 'regular', to the hidden patterns of the very large and very small. We should prize the ability to shift between these points of view, while carefully avoiding conflating them.
- Spell 2: Reality is normal. So it shouldn't be difficult for me to think like it.
- Counterspell: What's normal from a God's-eye view is wont to be weird from a human's. And vice versa.
Thinking like reality requires understanding reality. You can't just will yourself into becoming a high-fidelity map. If nothing else, you'll be too vulnerable to knowledge gaps. Pretending you think like quantum physics is even worse than rebelling against the physics for being too confusing. At least in the latter case you've noticed the disparity between your map and the territory.
To take pride in one's confusion at the quantum, rather than striving mightily to understand, is an epistemic sin. But to deny one's confusion at the quantum, to push it from one's mind and play-act at wisdom, is a graver sin. The goal isn't to do away with one's confusion; it's to do away with one's best reasons to be confused.
What adds up to the familiar needn't be familiar.
- Spell 3: It all adds up to normality. But relativity sure sounds abnormal! So relativity will be replaced by something normaler.
- Counterspell: Physics is the weird addenda, not the normal sum.
x + y + z + ... = Normality. But x ≠ Normality. Nor does y. In fact, even the complete quantum-mechanical description of the human world would not look normal, from a human perspective. But it would be a description of exactly the world we live in.
This is another reminder that surprise is a feature of maps, not of territories. Very different maps — a wave function scribbled in silicon, a belief, a gesture, a child's drawing — can represent the same territory. The drawing is normal (familiar), and the wave function isn't. But the referent of the two maps may well be the same. In asserting "quantum mechanics adds up to normality", we're really asserting that our maps of familiar objects, to the extent they're accurate, would co-refer with portions of the maps of an ideal Finished Science. They're two different languages, but faithful translation is possible.
- Spell 4: It all adds up to normality. So I should be able to readily intuit how QM yields the familiar.
- Counterspell: Reality forms what's normal, but by weird means.
The relationship between the quantum and the familiar can be thoroughly unfamiliar. The way in which the unintuitive fundamental truths yield an intuitive Middle World is not itself intuitive.
We could call this "emergence", if we wished to craft a territory predicate out of mapstuff. Otherwise, I suggest calling this predicate"wow-sometimes-I'm-not-very-good-at-keeping-track-of-lots-of-small-things-ence". Understanding the logical or mathematical implications of simple patterns is not humanity's specialty. Causal and part-whole relations are no exception to this rule.
It all adds up to the phenomenon.
- Spell 5: It all adds up to normality. It's common sense that time 'flows'. So science will ultimately show time is not at all like space.
- Counterspell: We'd find even the familiar alien, if we but understood it.
Normality isn't common sense. Normality isn't our beliefs, expectations, assumptions, or strongest convictions. Normality, in the sense relevant to a true and binding "It all adds up to normality", is the phenomenon, the human world as it appears. (Not the human world as it is believed to appear. The human world as we actually encounter it.)
What's "the phenomenon"?
I'm going to be willfully cagey about that. What I really mean by "the phenomenon" is "whatever's going on". Or whatever's going on that we're cognitively accessing or representing. It needn't all add up to a world that makes my map true; but it will add up to a world that explains why my map looks the way it does.
Even saying that much risks overly restricting the shape explanations are allowed to take. Egan's Law can be used as a general constraint on successful explanations — they must account for the explanandum — but only if we treat the explanandum loosely enough to permit dissolutions and eliminations alongside run-of-the-mill reductions. It all adds up to an explanation of how our beliefs arose, though not necessarily a validation of them.
- Spell 6: It all adds up to normality. I have an immediate epistemic acquaintance with the irreducible phenomenal character of my experience. So, whatever be the final theory, it will certainly include qualia.
- Counterspell: Every map is in a territory. But no meta-map is beyond suspicion.
It's obvious that appearances can deceive us about underlying reality. But a variety of perceptual illusions demonstrate that experiences can also consistently mislead us about themselves. It all adds up to normality, but normality lies.
In some cases, we can't accurately describe our surface impressions until we've understood their underlying mechanism. Since so much of science is revisionary, we mustn't interpret Egan's Law in a way that unduly privileges first-pass descriptions. When data and theory conflict, it's sometimes more likely that the data has been misrepresented than that the theory is false.
How far does this openness to map revision extend? As far as the dynamics of one's brain allows. From the sidelines, it may appear to me that in principle a thinker should be able to have certainty about some propositions — for instance, 'an experience is occurring'. But when I actually find myself living through such a thought, I don't in fact experience infinite confidence. It remains physically possible for me to be persuaded otherwise.
This point ought to be a bit controversial, and more defense of it is needed. But I do insist on treating 'phenomenal experience' and 'phenomenon' as prima facie independent concepts. Egan's Law really is the law, whereas claims about some inerrant mode of reasoning or perception will at best qualify as well-supported hypotheses.
Egan's Law is not about saving conscious experience, or our theories, or our axioms, or our interpretations or descriptions of the phenomenon. It's about saving the phenomenon itself — the piece of the world in which we are, in fact, submerged. Egan's Law can be restated: The part of reality that (under its familiar description) puzzled or surprised us is identical to (or otherwise lawfully derivable from) the part of reality that (under its more fundamental description) explains it.
The human piece of the universe is of a piece with everything else. And the Everything Else gets explanatory priority. Of all our science's findings to date, that may well be the most startling.
The following will explore a couple of areas in which I feel that the criminal justice system of many Western countries might be deficient, from the standpoint of rationality. I am very much interested to know your thoughts on these and other questions of the law, as far as they relate to rational considerations.
Moral luck refers to the phenomenon in which behaviour by an agent is adjudged differently based on factors outside the agent's control.
Suppose that Alice and Yelena, on opposite ends of town, drive home drunk from the bar, and both dazedly speed through a red light, unaware of their surroundings. Yelena gets through nonetheless, but Alice hits a young pedestrian, killing him instantly. Alice is liable to be tried for manslaughter or some similar charge; Yelena, if she is caught, will only receive the drunk driving charge and lose her license.
Raymond, a day after finding out that his ex is now in a relationship with Pardip, accosts Pardip at his home and attempts to stab him in the chest; Pardip smashes a piece of crockery over Raymond's head, knocking him unconscious. Raymond is convicted of attempted murder, receiving typically 3-5 years chez nous (in Canada). If he had succeeded, he would have received a life sentence, with parole in 10-25 years.
Why should Alice be punished by the law and demonized by the public so much more than Yelena, when their actions were identical, differing only by the sheerest accident? Why should Raymond receive a lighter sentence for being an unsuccessful murderer?
Some prima facie plausible justifications:
- Identical behaviour is hard to judge - perhaps Yelena was really keeping a better eye on the road than Alice; perhaps Raymond would have performed a non-fatal stabbing.
- The law needs to crack down harder when there are actual victims, in order to provide the victims and families a sense of justice done.
- This could result in far too many serious, high-level trials.
Trial by Jury; Trial by Judge
Those of us who like classic films may remember 12 Angry Men (1957) with Henry Fonda. This was a remarkably good film about a jury deliberating on the murder trial of a poor young man from a bad neighbourhood, accused of killing his father. It portrays the indifference (one juror wants to be out in time for the baseball game), prejudice and conformity of many of the jurors, and how this is overcome by one man of integrity who decides to insist on a thorough look through the evidence and testimony.
I do not wish to generalize from fictional examples; however, such factors are manifestly at play in real trials, in which Henry Fonda cannot necessarily be relied upon to save the day.
Komponisto has written on the Knox case, in which an Italian jury came to a very questionable (to put it mildly) conclusion based on the evidence presented to them; other examples will doubtless spring to mind (a famous one in this neck of the woods is the Stephen Truscott case - the evidence against Truscott being entirely circumstantial.
More information on trial by jury and its limitations may be found here. Recently the UK has made some moves to trial by judge for certain cases, specifically fraud cases in which jury tampering is a problem.
The justifications cited for trial by jury typically include the egalitarian nature of the practice, in which it can be guaranteed that those making final legal decisions do not form a special class over and above the ordinary citizens whose lives they effect.
A heartening example of this was mentioned in Thomas Levenson's fascinating book Newton and the Counterfeiter. Being sent to Newgate gaol was, infamously in the 17th and 18th centuries, an effective death sentence in and of itself; moreover, a surprisingly large number of crimes at this time were capital crimes (the counterfeiter whom Newton eventually convicted was hanged). In this climate of harsh punishment, juries typically only returned guilty verdicts either when evidence was extremely convincing or when the crime was especially heinous. Effectively, they counteracted the harshness of the legal system by upping the burden of proof for relatively minor crimes.
So juries sometimes provide a safeguard against abuse of justice by elites. However, is this price for democratizing justice too high, given the ease with which citizens naive about the Dark Arts may be manipulated? (Of course, judges are by no means perfect Bayesians either; however, I would expect them to be significantly less gullible.)
Are there any other systems that might be tried, besides these canonical two? What about the question of representation? Does the adversarial system, in which two sides are represented by advocates charged with defending their interests, conduce well to truth and justice, or is there a better alternative? For any alternatives you might consider: are they naive or savvy about human nature? What is the normative role of punishment, exactly?
How would the justice system look if LessWrong had to rewrite it from scratch?
Over this past weekend I listened to an episode of This American Life titled Pro Se. Although the episode is nominally about people defending themselves in court, the first act of the episode was about a man who pretended to act insane in order to get out of a prison sentence for an assault charge. There doesn't appear to be a transcript, so I'll summarize here first.
A man, we'll call him John, was arrested in the late 1990s for assaulting a homeless man. Given that there was plenty of evidence to prove him guilty, he was looking for a way to avoid the likely jail sentence of five to seven years. The other prisoners he was being held with suggested that he plead insanity: he'd be put up at a hospital for several months with hot food and TV and released once they considered him "rehabilitated". So he took bits and pieces about how insane people are supposed to act from movies he had seen and used them to form a case for his own insanity. The court believed him, but rather than sending him to a cushy hospital, they sent him to a maximum security asylum for the criminally insane.
Within a day of arriving, John realized the mistake he had made and sought to find a way out. He tries a variety of techniques: engaging in therapy, not engaging in therapy, dressing like a sane person, acting like a sane person, acting like an incurably insane person, but none of it works. Over a decade later he is still being held.
As the story unravels, we learn that although John makes a convincing case that he faked his way in and is being held unjustly, the psychiatrists at the asylum know that he faked his way in and continue to hold him anyway, though John is not aware of this. The reason: through his long years of documented behavior John has made it clear to the psychiatrists that he is a psychopath/sociopath and is not safe to return to society without therapy. John is aware that this is his diagnosis, but continues to believe himself sane.
Similar to trying to determine if you are anosognosic, how do you determine if you are insane? Some kinds of insanity can be self diagnosed, but in John's case he has lots of evidence (he has access to read all of his own medical records) that he is insane, yet continues to believe himself not to be. To me this seems a level trickier than anosognosis, since there's no physical tests you can make, but perhaps it's only a level of difference significant to people but not to an AI.
Edited to add a footnote: By "sane" I simply mean normative human reasoning: the way you expect, all else being equal, a human to think about things. While the discussion in the comments about how to define sanity might be of some interest, it really gets away from the point of the post unless you want to argue that "sanity" is creating a question here that is best solved by dissolving the question (as at least one commenter does).
(Inspired by a recent conversation with Robin Hanson.)
Robin Hanson, in his essay on "Minimal Morality", suggests that the unreliability of our moral reasoning should lead us to seek simple moral principles:
"In the ordinary practice of fitting a curve to a set of data points, the more noise one expects in the data, the simpler a curve one fits to that data. Similarly, when fitting moral principles to the data of our moral intuitions, the more noise we expect in those intuitions, the simpler a set of principles we should use to fit those intuitions. (This paper elaborates.)"
In "the limit of expecting very large errors of our moral intuitions", says Robin, we should follow an extremely simple principle - the simplest principle we can find that seems to compress as much morality as possible. And that principle, says Robin, is that it is usually good for people to get what they want, if no one else objects.
Now I myself carry on something of a crusade against trying to compress morality down to One Great Moral Principle. I have developed at some length the thesis that human values are, in actual fact, complex, but that numerous biases lead us to underestimate and overlook this complexity. From a Friendly AI perspective, the word "want" in the English sentence above is a magical category.
But Robin wasn't making an argument in Friendly AI, but in human ethics: he's proposing that, in the presence of probable errors in moral reasoning, we should look for principles that seem simple to us, to carry out at the end of the day. The more we distrust ourselves, the simpler the principles.
This argument from fitting noisy data, is a kind of logic that can apply even when you have prior reason to believe the underlying generator is in fact complicated. You'll still get better predictions from the simpler model, because it's less sensitive to noise.
Even so, my belief that human values are in fact complicated, leads me to two objections and an alternative proposal:
Suppose that your good friend, the police commissioner, tells you in strictest confidence that the crime kingpin of your city is Wulky Wilkinsen. As a rationalist, are you licensed to believe this statement? Put it this way: if you go ahead and mess around with Wulky's teenage daughter, I'd call you foolhardy. Since it is prudent to act as if Wulky has a substantially higher-than-default probability of being a crime boss, the police commissioner's statement must have been strong Bayesian evidence.
Our legal system will not imprison Wulky on the basis of the police commissioner's statement. It is not admissible as legal evidence. Maybe if you locked up every person accused of being a crime boss by a police commissioner, you'd initially catch a lot of crime bosses, plus some people that a police commissioner didn't like. Power tends to corrupt: over time, you'd catch fewer and fewer real crime bosses (who would go to greater lengths to ensure anonymity) and more and more innocent victims (unrestrained power attracts corruption like honey attracts flies).
This does not mean that the police commissioner's statement is not rational evidence. It still has a lopsided likelihood ratio, and you'd still be a fool to mess with Wulky's teenager daughter. But on a social level, in pursuit of a social goal, we deliberately define "legal evidence" to include only particular kinds of evidence, such as the police commissioner's own observations on the night of April 4th. All legal evidence should ideally be rational evidence, but not the other way around. We impose special, strong, additional standards before we anoint rational evidence as "legal evidence".
As I write this sentence at 8:33pm, Pacific time, on August 18th 2007, I am wearing white socks. As a rationalist, are you licensed to believe the previous statement? Yes. Could I testify to it in court? Yes. Is it a scientific statement? No, because there is no experiment you can perform yourself to verify it. Science is made up of generalizations which apply to many particular instances, so that you can run new real-world experiments which test the generalization, and thereby verify for yourself that the generalization is true, without having to trust anyone's authority. Science is the publicly reproducible knowledge of humankind.
Like a court system, science as a social process is made up of fallible humans. We want a protected pool of beliefs that are especially reliable. And we want social rules that encourage the generation of such knowledge. So we impose special, strong, additional standards before we canonize rational knowledge as "scientific knowledge", adding it to the protected belief pool.
View more: Next