Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

Firewalling the Optimal from the Rational

86 Eliezer_Yudkowsky 08 October 2012 08:01AM

Followup to: Rationality: Appreciating Cognitive Algorithms  (minor post)

There's an old anecdote about Ayn Rand, which Michael Shermer recounts in his "The Unlikeliest Cult in History" (note: calling a fact unlikely is an insult to your prior model, not the fact itself), which went as follows:

Branden recalled an evening when a friend of Rand's remarked that he enjoyed the music of Richard Strauss. "When he left at the end of the evening, Ayn said, in a reaction becoming increasingly typical, 'Now I understand why he and I can never be real soulmates. The distance in our sense of life is too great.' Often she did not wait until a friend had left to make such remarks."

Many readers may already have appreciated this point, but one of the Go stones placed to block that failure mode is being careful what we bless with the great community-normative-keyword 'rational'. And one of the ways we do that is by trying to deflate the word 'rational' out of sentences, especially in post titles or critical comments, which can live without the word.  As you hopefully recall from the previous post, we're only forced to use the word 'rational' when we talk about the cognitive algorithms which systematically promote goal achievement or map-territory correspondences.  Otherwise the word can be deflated out of the sentence; e.g. "It's rational to believe in anthropogenic global warming" goes to "Human activities are causing global temperatures to rise"; or "It's rational to vote for Party X" deflates to "It's optimal to vote for Party X" or just "I think you should vote for Party X".

If you're writing a post comparing the experimental evidence for four different diets, that's not "Rational Dieting", that's "Optimal Dieting". A post about rational dieting is if you're writing about how the sunk cost fallacy causes people to eat food they've already purchased even if they're not hungry, or if you're writing about how the typical mind fallacy or law of small numbers leads people to overestimate how likely it is that a diet which worked for them will work for a friend. And even then, your title is 'Dieting and the Sunk Cost Fallacy', unless it's an overview of four different cognitive biases affecting dieting. In which case a better title would be 'Four Biases Screwing Up Your Diet', since 'Rational Dieting' carries an implication that your post discusses the cognitive algorithm for dieting, as opposed to four contributing things to keep in mind.

continue reading »

The peril of ignoring emotions

15 Swimmer963 03 April 2011 05:15PM

Related to: Luminosity Sequence, Unknown Knowns

Let me introduce you to a hypothetical high school student, Sally. She’s smart and pretty and outgoing, and so are her friends. She considers herself a modern woman, sexually liberated, and this is in line with the lifestyle her friends practice. They think sex is normal and healthy and fun. Sally isn’t just pretending in order to fit in; these really are her friends, this really is her milieu, and according to health class, sex between consenting adults is nothing to be ashamed of. Sally isn't a rigorous rationalist, although she likes to think of herself as rational, and she's no more self-aware than the average high school girl. 

Now Sally meets a boy, Bob, and she things he’s cute, and he thinks she’s cute too. Bob is part of her crowd. Her friends like him; he respects women and treats Sally well and, like any healthy teenage boy, fairly horny. According to her belief system, that shouldn’t set off any alarm bells. She’s been warned about abusive relationships, but Bob is a nice guy. So when they go upstairs together at her friend’s party, she has every reason to be excited and a little nervous, but not uncomfortable. The idea that Mom wouldn’t approve is so obviously irrelevant that she ignores it completely.

...And afterwards, she feels guilty and violated and horrible about herself, even though it was her decision.

I used this example because I expect it’s not unusual. On the surface, Sally’s discomfort seems to come out of nowhere, but modern North American society is chock-full of contradictory beliefs about sex. Sex is normal and healthy. Sex is dirty. Sex is only for when you’re married. If Sally’s mother is Christian, or even just conservative, Sally would have internalized those beliefs when she was a child. It would have been hard not to. They’re her unknown knowns, and she may not have noticed them before, because there’s a wide psychological gap between believing it’s okay for others to behave a particular way, and believing it’s okay for you. The meme ‘don’t pass judgement on other people’ is, I think, pretty widespread in North America and maybe more so in Canada, but so is holding oneself to a high standard...and those are contradictory.

I think that the nagging, seemingly irrational moment of ‘that doesn’t feel right’ is important. It potentially reveals something about the beliefs and attitudes you hold that you don’t even know about. Sally’s response to her nagging doubt could have been the following:

Hmm, that’s interesting, why does it bother me so much that Mom would disapprove? I guess when we used to go to church, they said sex was only for when you’re married. But I don’t believe anything else they said in church. ...Well, I guess I want Mom to be proud of me. I want her to praise me for doing well in school. And I think lying is wrong, so the fact that I either have to lie to her about having had sex, or face her disapproval, maybe that’s why I’m uncomfortable? But I don’t want to say no, it’ll make me look like a prude... Still, what if everyone feels this way at the start? I know Alice went to church too when she was a kid, and her mom would kill her if she knew she was sexually active, I wonder if that bothers Alice? Hmm, I think maybe it’s still the right choice to sleep with Bob, but maybe I’m taking this too lightly? Maybe this should be a big deal and I should feel anxious? After all, he might judge me anyway, he might think I’m too easy, or a slut. Maybe I can just explain to him that I want to think about this longer... After all, why should I assume something is right just because they told us in health class? That’s just like in church, it’s taking someone else’s opinion on faith. I’ve never actually thought about this, I’ve just followed other people. Who’s to say they’re right?

Whatever decision Sally makes, she probably won’t feel violated. She listened to her feelings and took them into consideration, even though they seemed irrational. As it turned out, they were a reasonable consequence of a belief-fragment that she hadn’t even known she had. So as a consequence of stopping to think, she knows herself better too. She’ll be better able to predict her behaviour in future situations. She’ll be less likely to ignore her threshold-warning discomfort and make risky choices as a result of peer pressure alone. She’ll be more likely to think.

To conclude: emotions exist. They are real. If you ignore them and plow on ahead, you won’t necessarily thank yourself afterwards. And that nagging feeling is a priceless moment to find out about your unknown knowns...which may not be rational, which may have been laid down in some previous era and never questioned since, but which part of you is going to try to uphold until you consciously deconstruct them. 

Rational Reading: Thoughts On Prioritizing Books

27 patrissimo 27 March 2011 07:54PM

A large element of instrumental rationality consists of filtering, prioritizing, and focusing.  It's true for tasks, for emails, for blogs, and for the multitude of other inputs that many of us are drowning in these days[1].  Doing everything, reading everything, commenting on everything is simply not an option - it would take infinite time.  We could simply limit time and do what happens to catch our attention in that limited time, but that's clearly not optimal.  Spending some time prioritizing rather than executing will always improve results if items can be prioritized and vary widely in benefit.  So maximizing the results we get from our finite time requires, for a variety of domains:

  1. Filtering: a quick first-pass to get input down to a manageable size for the higher-cost effort of prioritizing.
  2. Prioritizing: briefly evaluating the impact each item will have towards your goals.
  3. Focusing: on the highest-priority items.

I have some thoughts, and am looking for more advice on how to do this for non-fiction reading.  I've stopped buying books that catch my attention, because I have an inpile of about 3-4 shelves of unread books that have been unread for years.  Instead, I put them on my Amazon Wishlists, which as a result have swelled to a total of 254 books - obviously un-manageable, and growing much faster than I read.

One obvious question to ask when optimizing is: what is the goal of reading?  Let me suggest a few possibilities:

  • Improve performance at a current job/role.  For example, as Executive Director of a nonprofit, I could read books on fundraising or management.
  • Relatedly, work towards a current goal.  Here is where it helps to have identified your goals, perhaps in an Annual Review.  As a parent, for example, there are an infinitude of parenting books that I could read, but I chose for this year to work specifically on positive psychology parenting, as it seemed like a potentially high-impact skill to learn.  This massively filters the set of possible parenting books.  Essentially, goal-setting ("learn positive psychology parenting habits") was a conscious prioritization step based on considering what new parenting skills would best advance my goals (in this case, to benefit my kids while making parenting more pleasant along the way).
  • Improve core skills or attributes relevant to many areas of life - productivity, happiness, social skills, diet, etc.
  • Expand your worldview (improve your map).  Myopically focusing only on immediate needs would eliminate some of the greatest benefit I feel I've gotten from non-fiction in my life, which is a richer and more accurate understanding of the world.
  • Be able to converse intelligently on currently popular books.  (Much as one might watch the news in order to facilitate social bonding by being able to discuss current events).  Note that I don't actually recommend this as a goal - I think you can find other things to bond over, plus you will sometimes read currently popular books because they serve other goals - but it may be important for some people.
continue reading »

A Rationalist's Bookshelf: The Mind's I (Douglas Hofstadter and Daniel Dennett, 1981)

15 colinmarshall 26 August 2009 07:08PM

When the call to compile a reading list for new rationalists went out, contributor djcb responded by suggesting The Mind's I: Fantasies and Reflections on Self and Soul, a compilation of essays, fictions and excerpts "composed and arranged" by Douglas Hofstadter and Daniel Dennett. Cut to me peering guiltily over my shoulder, my own copy sitting unread on the shelf, peering back.

The book presents Hofstadter and Dennett's co-curation of 27 pieces, some penned by the curators themselves, meant to "reveal" and "make vivid" a set of "perplexities," to wit: "What is the mind?" "Who am I?" "Can mere matter think or feel?" "Where is the soul?" Two immediate concerns arise. First, The Mind's I's 1981 publication date gives it access to the vast majority of what's been thought and said about these questions, but robs it of of any intellectual progress toward the answers made in the nearly three decades since. (This turns out not to be an issue, as most of the answers seem to have drawn no closer in the 1980s, 1990s or 2000s.) Second, those sound suspiciously similar to questions hazily articulated by college freshmen, less amenable to "rational inquiry" than to "dorm furniture and bad weed." They don't quite pass the "man test," an reversal of the fortune cookie "in bed" game: simply tack "man" onto the beginning of each question and see who laughs. "Man, who am I?" "Man, where is the soul?" "Man, can matter think or feel?"

Hofstadter and Dennett's fans know, however, that their analyses rise a cut above, engaged as they are in the admirable struggle to excise the navel-gazing from traditionally navel-gazey topics. The beauty is that they've always accomplished this, together and separately, not by making these issues less exciting but by making them more so. Their clear, stimulating exegeses, explorations and speculations brim with both the enthusiasm of the thrilled neophyte and the levelheadedness of the seasoned surveyor. They even do it humorously, Hofstadter with his zig-zaggy punniness and Dennett with his wit that somehow stays just north of goofy. Thus armed, they've taken on such potentially dangerous topics as whether words and thoughts follow rules, how the animate emerges from the inanimate (Hofstader's rightly celebrated Gödel, Escher, Bach: An Eternal Golden Braid) and consciousness (most of Dennett's career), on the whole safely.

But obviously this is not a "pure" (whatever that might mean) Hofstadter-Dennett joint; rather, their editorial choices compose one half and their personal commentaries — "reflections," they banner them — on the fruits of those choices compose the other. Nearly every selection, whether a short story, article, novel segment or dialogue, leads into an original discussion and evaluation by, as they sign them, D.R.H. and/or D.C.D. They affirm, they contradict, they expand, they question, they veer off in their own directions; the reflections would make a neat little book on the topics at hand by themselves.

continue reading »

A final thought

-1 psycho 20 March 2009 06:37PM

I would like to thank all of those people who voted my last post down into the negative numbers allowing me to exploit a flaw in this sites design. I was really getting bored of trying to figure a way to get a karma score of 20. So I toast all those who helped me achieve it in record time.

Now on to the meat of the issue, I thought when I ran into this group that it would be what it claimed a site devoted to refining rationality, I could not have been more wrong. This site is not concerned with rationality or improving it but rather self adulation. You people are not concerned with being rational just imagining you are rational and then congratulating yourselves on your own self deception. I will be totally honest here this little group therapy session you guys have going on is intellectually repugnant. You all should be ashamed of yourselves that you have nothing better to do then congratulate yourselves on non-accomplishments.

If any of you were truly intelligent you would feel no need to engage in this behavior and your leader would not be a man who dropped out of grade school. So enjoy your idol of rationality and self-proclaimed genius if any of you decide that rationality is of interest I suggest you try learning some math a feat that is beyond the abilities of your leader. Lest of any of you doubt it he has never published a technical paper. All but one of his papers is self published (so as to avoid critique) and the only math that he demonstrates knowledge of is freshman statistics.