The Shadow Question
This is part 2 of a sequence on problem solving. Here's part 1, which introduces the vocabulary of "problems" versus "tasks". This post's title is a reference1 worth 15 geek points if you get it without Googling, and 20 if you can also get it without reading the rest of the post.
You have to be careful what you wish for. You can't just look at a problem, say "That's not okay," and set about changing the world to contain something, anything, other than that. The easiest way to change things is usually to make them worse. If I owe the library fifty cents that I don't have lying around, I can't go, "That's not okay! I don't want to owe the library fifty cents!" and consider my problem solved when I set the tardy book on fire and now owe them, not money, but a new copy of the book. Or you could make things, not worse in the specific domain of your original problem, but bad in some tangentially related department: I could solve my library fine problem by stealing fifty cents from my roommate and giving it to the library. I'd no longer be indebted to the library. But then I'd be a thief, and my roommate might find out and be mad at me. Calling that a solution to the library fine problem would be, if not an outright abuse of the word "solution", at least a bit misleading.
So what kind of solutions are we looking for? How do we answer the Shadow Question? It's hard to turn a complex problem into doable tasks without some idea of what you want the world to look like when you've completed those tasks. You could just say that you want to optimize according to your utility function, but that's a little like saying that your goal is to achieve your goals: no duh, but now what? You probably don't even know what your utility function is; it's not a luminous feature of your mind.
MWI, weird quantum experiments and future-directed continuity of conscious experience
Response to: Quantum Russian Roulette
Related: Decision theory: Why we need to reduce “could”, “would”, “should”
In Quantum Russian Roulette, Christian_Szegedy tells of a game which uses a "quantum source of randomness" to somehow make a game which consists in terminating the lives of 15 rich people to create one very rich person sound like an attractive proposition. To quote the key deduction:
Then the only result of the game is that the guy who wins will enjoy a much better quality of life. The others die in his Everett branch, but they live on in others. So everybody's only subjective experience will be that he went into a room and woke up $750000 richer.
I think that Christian_Szegedy is mistaken, but in an interesting way. I think that the intuition at steak here is something about continuity of conscious experience. The intuition that Christian might have, if I may anticipate him, is that everyone in the experiment will actually experience getting $750,000, because somehow the word-line of their conscious experience will continue only in the worlds where they do not die. To formalize this, we imagine an arbitrary decision problem as a tree with nodes corresponding to decision points that create duplicate persons, and time increasing from left to right:

The Lifespan Dilemma
One of our most controversial posts ever was "Torture vs. Dust Specks". Though I can't seem to find the reference, one of the more interesting uses of this dilemma was by a professor whose student said "I'm a utilitarian consequentialist", and the professor said "No you're not" and told them about SPECKS vs. TORTURE, and then the student - to the professor's surprise - chose TORTURE. (Yay student!)
In the spirit of always making these things worse, let me offer a dilemma that might have been more likely to unconvince the student - at least, as a consequentialist, I find the inevitable conclusion much harder to swallow.
Decision theory: Why we need to reduce “could”, “would”, “should”
(This is the second post in a planned sequence.)
Let’s say you’re building an artificial intelligence named Bob. You’d like Bob to sally forth and win many utilons on your behalf. How should you build him? More specifically, should you build Bob to have a world-model in which there are many different actions he “could” take, each of which “would” give him particular expected results? (Note that e.g. evolution, rivers, and thermostats do not have explicit “could”/“would”/“should” models in this sense -- and while evolution, rivers, and thermostats are all varying degrees of stupid, they all still accomplish specific sorts of world-changes. One might imagine more powerful agents that also simply take useful actions, without claimed “could”s and “woulds”.)
My aim in this post is simply to draw attention to “could”, “would”, and “should”, as concepts folk intuition fails to understand, but that seem nevertheless to do something important for real-world agents. If we want to build Bob, we may well need to figure out what the concepts “could” and “would” can do for him.*
Why You're Stuck in a Narrative
The narrative fallacy addresses our limited ability to look at sequences of facts without weaving an explanation into them, or, equivalently, forcing a logical link, an arrow of relationship upon them. Explanations bind facts together. They make them all the more easily remembered; they help them make more sense. Where this propensity can go wrong is when it increases our impression of understanding.
Knowing What You Know
From Kahneman and Tversky:
"A person is said to employ the availability heuristic whenever he estimates frequency or probability by the ease with which instances or associations could be brought to mind"
Confusion about Newcomb is confusion about counterfactuals
(This is the first, and most newcomer-accessible, post in a planned sequence.)
Newcomb's Problem:
Joe walks out onto the square. As he walks, a majestic being flies by Joe's head with a box labeled "brain scanner", drops two boxes on the ground, and departs the scene. A passerby, known to be trustworthy, comes over and explains...
If Joe aims to get the most money, should Joe take one box or two?
What are we asking when we ask what Joe "should" do? It is common to cash out "should" claims as counterfactuals: "If Joe were to one-box, he would make more money". This method of translating "should" questions does seem to capture something of what we mean: we do seem to be asking how much money Joe can expect to make "if he one-boxes" vs. "if he two-boxes". The trouble with this translation, however, is that it is not clear what world "if Joe were to one-box" should refer to -- and, therefore, it is not clear how much money we should say Joe would make, "if he were to one-box". After all, Joe is a deterministic physical system; his current state (together with the state of his future self's past light-cone) fully determines what Joe's future action will be. There is no Physically Irreducible Moment of Choice, where this same Joe, with his own exact actual past, "can" go one way or the other.
Fighting Akrasia: Finding the Source
Followup to: Fighting Akrasia: Incentivising Action
Influenced by: Generalizing From One Example
Previously I looked at how we might fight akrasia by creating incentives for actions. Based on the comments to the previous article and Yvain's now classic post Generalizing From One Example, I want to take a deeper look at the source of akrasia and the techniques used to fight it.
I feel foolish for not looking at this closer first, but let's begin by asking what akrasia is and what causes it. As commonly used, akrasia is the weakness-of-will we feel when we desire to do something but find ourselves doing something else. So why do we experience akrasia? Or, more to the point, why to we feel a desire to take actions contrary the actions we desire most, as indicated by our actions? Or, if it helps, flip that question and ask why are the actions we take not always the ones we feel the greatest desire for?
First, we don't know the fine details of how the human brain makes decisions. We know what it feels like to come to a decision about an action (or anything else), but how the algorithm feels from the inside is not a reliable way to figure out how the decision was actually made. But because most people can relate to a feeling of akrasia, this suggests that there is some disconnect between how the brain decides what actions are most desirable and what actions we believe are most desirable. The hypothesis that I consider most likely is that the ability to form beliefs about desirable actions evolved well after the ability to make decisions about what actions are most desirable, and the decision-making part of the brain only bothers to consult the belief-about-desirability-of-actions part of the brain when there is a reason to do so from evolution's point of view.1 As a result we end up with a brain that only does what we think we really want when evolutionarily prudent, hence we experience akrasia whenever our brain doesn't consider it appropriate to consult what we experience as desirable.
This suggests two main ways of overcoming akrasia assuming my hypothesis (or something close to it) is correct: make the actions we believe to be desirable also desirable to the decision-making part of the brain or make the decision-making part of the brain consult the belief-about-desirability-of-actions part of the brain when we want it to. Most techniques fall into the former category since this is by far the easier strategy, but however a technique works, an overriding theme of the akrasia-related articles and comments on Less Wrong is that no technique yet found seems to work for all people.
The Hero With A Thousand Chances
"Allow me to make sure I have this straight," the hero said. "I've been untimely ripped from my home world to fight unspeakable horrors, and you say I'm here because I'm lucky?"
Aerhien dipped her eyelashes in elegant acknowledgment; and quietly to herself, she thought: Thirty-seven. Thirty-seven heroes who'd said just that, more or less, on arrival.
Not a sign of the thought showed on her outward face, where the hero could see, or the other council members of the Eerionnath take note. Over the centuries since her accidental immortality she'd built a reputation for serenity, more or less because it seemed to be expected.
"There are kinds and kinds of luck," Aerhien said serenely. "Not every person desires their personal happiness above all else. Those who are lucky in aiding others, those whose luck is great in succor and in rescue, these ones are not always happy themselves. You are here, hero, because you have a hero's luck. The boy whose dusty heirloom sword proves to be magical. The peasant girl who finds herself the heir to a great kingdom. Those who discover, in time of sudden stress, an untrained wild magic within themselves. Success born not of learning, not of skill, not of determination, but unplanned coincidence and fortunes of birth: That is a hero's luck."
Information cascades in scientific practice
Here's an interesting recent paper in the British Medical Journal: "How citation distortions create unfounded authority: analysis of a citation network". (I don't know if this is freely accessible, but the abstract should be.)
From the paper:
"Objective To understand belief in a specific scientific claim by studying the pattern of citations among papers stating it."
"Conclusion Citation is both an impartial scholarly method and a powerful form of social communication. Through distortions in its social use that include bias, amplification, and invention, citation can be used to generate information cascades resulting in unfounded authority of claims. Construction and analysis of a claim specific citation network may clarify the nature of a published belief system and expose distorted methods of social citation."
It also includes a list of specific ways in which citations were found to amplify or invent evidence.
View more: Next
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)