Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

SotW: Be Specific

39 Post author: Eliezer_Yudkowsky 03 April 2012 06:11AM

(The Exercise Prize series of posts is the Center for Applied Rationality asking for help inventing exercises that can teach cognitive skills.  The difficulty is coming up with exercises interesting enough, with a high enough hedonic return, that people actually do them and remember them; this often involves standing up and performing actions, or interacting with other people, not just working alone with an exercise booklet and a pencil.  We offer prizes of $50 for any suggestion we decide to test, and $500 for any suggestion we decide to adopt.  This prize also extends to LW meetup activities and good ideas for verifying that a skill has been acquired.  See here for details.)

Exercise Prize:  Be Specific

During YCombinator's Startup School 2011, Paul Graham and Harj Tagger did "office hours" onstage.  One pair of entrepreneurs were doing a matchmaking (dating) startup, and Paul and Harj were trying to figure out what their startup did, exactly - for example, what their startup could do that the existing low-tech solution couldn't.  (Video.)

Harj:  Low-tech like, you know, just like word of mouth, telling someone "hey, you should like, meet up with my friend" or "we're getting drinks, why don't you come along?" Like, what can the software do that's specifically better than that?

Entrepreneur:  I think that our software specifically is providing the better connections for people, um...

Paul: Providing the better connections for people...?

Entrepreneur:  I mean, one way you can think about it, I don't know if this is the right answer, but... there's a lot of things that are happening in real life that they're trying to mimic online, maybe that's not the correct way to...  Look at it like this: to give them an online tool to also do this, like they're already doing in real life, maybe they could reach, uh expand their reach through the online website.

This had been happening with most of the startups Paul and Harj were interrogating - they just could not seem to provide a customer use-case - and I couldn't stand it any more; which is why at this point I whispered audibly enough for a few nearby people to hear, "Be specific!  Be specific!"

A moment later, on stage:

Paul:  Hm.  Not very specific.

I got some strange looks from the people sitting next to me.

I hope this provides some background for my guess that around half of Paul Graham's advantage is based on years of incubator experience, and the other half is unusual rationality skills of the sort that the Center for Modern Rationality is trying to figure out how to teach.  Obviously this is only a very rough conjecture.  But you can see the basis for the hope that - after a fair amount more work - we'll be able to offer a 2-day course for YCombinator entrepreneurs that eliminates 50% of the overhead from their conversations with Paul Graham.

(Also, note how this post starts off with a specific example - an instance of the concrete-abstract writing pattern in which you state the example first and the generalization afterward.  This is one of the most common bits of nonfiction writing advice I dispense:  "Open with the concrete example, not the abstract explanation!")

Theoretical background:

S. I. Hayakawa once gave this illustration of the "ladder of abstraction", and in particular, the difference between going up or down:

"What is meant by the word red?"
"It's a color."
"What's a color?"
"Why, it's a quality things have."
"What's a quality?"


"What is meant by the word red?"
"Well, the next time you see some cars stopped at an intersection, look at the traffic light facing them.  Also, you might go to the fire department and see how their trucks are painted."

"Red is a color" is moving up the ladder; "color" is a supercategory of red.  All things which are red, have colors; but not all things which have colors, are red.  And similarly, if you look at a specific firetruck, that firetruck is a red thing, but there are also many other red things which are not that firetruck.

What is true of one apple may not be true of another apple; suppose apple1 weighs 100 grams and is slightly green in some places, and apple2 weighs 200 grams and is entirely dark-red.  You can say more truths about apple2, like "apple2 is dark red", then you can say that is true of all apples.  (For more on this point see The Virtue of Narrowness.)

Thus, it may be easier to mentally picture "a firetruck" than "something red" - "firetruck" describes a narrower section of Thingspace, so you're less likely to get lost along the way.

S. I. Hayakawa called this the ladder of abstraction.  I'm not sure if understanding the following section will really help with the skill of Being Specific, or help anyone construct exercises for the skill of being specific.  But a better theoretical understanding does sometimes prove useful.  So I will now digress to explain that abstraction isn't really a ladder, but a lattice.

Let's illustrate this using a classic example from the field of machine learning.  Suppose that Days have three properties:

  • Weather: {Sunny, Cloudy, Rainy}
  • Temperature: {Cool, Hot}
  • Timing: {Weekday, Weekend}

And suppose that we've been given some examples of Days on which it was good, or alternatively bad, to play tennis.  For example, the Day {Sunny, Cool, Weekend} was good for playing tennis, but the day {Rainy, Hot, Weekday} was bad for playing tennis.  A classic task in machine learning is to induct, from a set of pre-classified examples like these, a rule describing when it is good to play tennis.

Any proposed rule which can classify all days as good or bad is a concept, in the lingo of machine learning.  "Sunny Days" is a concept; likewise "Sunny Cool Days", and "Days which are either Cool or Sunny".  Each of these is a concept which classifies all 12 possible days either positively or negatively - instances or non-instances of the concept.

There are 212 possible concepts over the 12 possible Days.  Why so many?  Because - for example - there's a concept which only includes the two Days {Sunny+Cool+Weekday} and {Cloudy+Cool+Weekend}}, but classifies all other Days as noninstances.  This is a way of classifying all Days into instances or noninstances, hence a possible concept.  It's not a compact concept, but it's a concept.  Each Day can be classified either positively or negatively - one binary decision per Day - so 212 possible concepts.  (That's why induction is a difficult problem in machine learning.)

The concept "Sunny" is a superconcept of "Sunny and Cool"; it lies above it in the lattice of abstraction, since all days which are "Sunny and Cool" are "Sunny".  "Sunny or Hot" is a supercategory of "Sunny".  "Weekend" is neither a superconcept nor a subconcept of "Sunny".

Concepts form a directed lattice from most general to most specific, with "all Days" at the top (every Day classified as an instance) and "no Days" at the bottom (the concept which classifies every Day as a noninstance).

If you now go back to the problem of telling someone what "red" means, when you say "red is a color", then, even if the listener does happen to know what "color" means, you're still moving upward in the lattice of abstraction.  When you said "color", you were talking about a concept that included all red things, but also many other things that were not red.

"Our software is providing the better connections for people" - the entrepreneur who said that might have had something specific in mind, or they might have just been bluffing or succumbing to wishful thinking.  But they described it using an abstract statement so broad that it included Facebook, or Western Union back when they were sending telegrams.  They might - though this is somewhat optimistic - they might have known themselves what they had in mind; they didn't think of Facebook; so they didn't realize how many other possibilities fit their words.  This is a classic manifestation of the Illusion of Transparency, and it's why we have to keep telling people to navigate the lattice downward.

The skill of Being Specific is the skill of understanding how to navigate the lattice of abstraction.  You can see why this would be a key element of cognition on a par with Bayes's Theorem or consequentialism.

And this is true in practice as well as theory.  When I'm talking to anyone outside the local LW community, I find that a very large amount of my conversation involves repeatedly asking them to be more specific - and if you think that's just me being annoying, watch Paul Graham in the video.

A closely related skill is concreteness, which has to do with nearness-to-sensory-experience or actionability.

According to David Allen's "Getting Things Done", for your brain to stop thinking about an unfinished task, you must (1) know and trust that an external system will remind you to perform that task when it is time to perform it, and (2) have chosen the next action taken at a sufficiently concrete level that your brain is no longer trying to plan it out in the background.  "Contact Luke about dispersing prize awards" is not a sufficiently concrete to-do; it leaves open the question of whether to phone or email, and what exactly to say.  "Read through the comments, gather the LessWrong usernames of everyone who made a suggestion we tried or adopted, and email the list to Luke" is an action item I know how to perform straightforwardly, without my brain trying to plan it in the background.  When you have a trustworthy external system to remind you of what to do, at the time you need to do it - so that the back of your mind isn't worrying about remembering to check the to-do list - and all to-do items have been concretized to the point of being executable without further background planning - then you have, in GTD parlance, "gotten to zero", a state of pure mental blissfulness in which your brain is not worrying about anything except what you're doing right now.

Similarly, for a statement like "Wulky Wilkinsen is a post-utopian" or "Earth gravity pulls at 9.8 meters per second squared" to be falsifiable, it must be concretized - rendered near-to-experience - to a sufficient degree that you can potentially see something and say "Oh, guess the hypothesis was wrong"; you must be able to have an experience which the concretized statement constrains, and which falsifies the theory if the experience is out-of-bounds.

Theoretically:  If you imagine the universe as a huge directed graph of causes and effects - the Great Web of Causality - then "concreteness" is being near enough in the Web to either your sensory inputs or motor outputs that you can directly see the prediction unfold, or directly implement the plan, without much further thought.

"Be Specific" and "Be Concrete" could easily end up being the same unit - they're closely related - and we're happy to entertain exercises for Being Concrete, as well as Being Specific.  Visualizing what your customer literally sees or does after navigating to your site, would've been a good first step toward being able to answer many of Paul Graham's questions.

A possible success criterion:

One question that we spent a lot of time discussing at CMR, was translating our sense of "specific enough" or "concrete enough" into a describable criterion.  (Instead of just a wordless intuition for when something is "too abstract".)

There was an exchange in Paul Graham's office hours that went like this, while interviewing a startup that did metrics - analyzing pageviews, roughly - and the entrepreneur was having great trouble describing what they did that MixPanel didn't.  It went on for a while.  It was painful to watch.

Paul:  I don't get what the difference is.  I still don't get what the difference is.  What's the difference between you and MixPanel?

Entrepreneur:  The difference is - when you have to supplement - they're a view company and we're a platform.  That's what it comes down to.  They're like a view, a reporting company.  If you need something they don't have, a feature - 

Harj:  So what's an example of somewhere you'd use your thing over MixPanel?  Can you give a use-case?

Entrepreneur:  Yeah, I mean, we had revenue on day zero. There's a good reason for um... it's a start up, it's a series A company in the daily deals space.  One we've signed a social game company to -

Harj:  And why do they prefer your thing?

Paul:  That wasn't what Harj was asking.

The problem (from the perspective of our present discussion) is that the Entrepreneur did not understand that Paul and Harj were repeatedly asking him to move downward on the ladder of abstraction.  When the Entrepreneur said "We had revenue on day zero", he was trying to offer confirmation of the abstract statement "We can do things MixPanel can't", but Paul and Harj still had no idea what his startup actually did.[1]

A quick bit of theoretical background:  There's an important difference, in the field of mathematical logic, between models and axioms.  An axiom is something like "All kittens are cute", i.e. "All x: kitten(x)->cute(x)".  A model is a particular universe of objects that includes {Obj #19834, kitten: T, cute: T, color: grey} and {Obj #19835, kitten: F, cute: F, color: striped}, and so on.

Correspondingly, in logical inference, there's a distinction between model-checking and deduction.  Suppose you want to know whether it's true that all positive integers less than 5, when multiplied by 7, are less than 50.  If you prove the general truth that all integers less than 5, times 7, are less than 35, by manipulating the axioms of multiplication and inequality, that's deduction.  If you notice that the only positive integers less than 5 are just {1, 2, 3, 4} and enumerate their products {7, 14, 21, 28}, which are all less than 50, that's model-checking.

My hypothesis about what it means to be "specific enough" or "concrete enough" is that the picture painted is detailed enough to use in model-checking whatever points are being debated.  Paul and Harj don't want to trust you when you state the abstract generalization, "We're better than MixPanel".  They aren't even content with deducing support for this generalization from the further generalization, "We already have customers."  They want a picture of something you do that MixPanel doesn't, which is detailed enough that they can model-check whether you have a competitive advantage.

Not to mention that Paul Graham is probably thinking about a number of other questions:

  • How much would I pay for this product?
  • Is this startup exciting enough that I would tweet about using it?
  • How much resources will it take to develop these features further?

Paul Graham doesn't want you to say, "$50, yes, and twenty engineer-months".  He wants a sufficiently specific picture of (a customer using) your product that he can arrive at his own answers by model-checking.

If Paul Graham is reading this, he's welcome to contradict my interpretation of what was going on in that particular session - but it did seem like a very nice concrete illustration.

That's my guess for what often constitutes "specific enough" - though I'm not sure that's the only thing that ever determines specific-enoughness.

[1]:  The strange part was, near the end of that session, it started to look like this might be an interesting startup; that the Entrepreneur wasn't just bluffing.  Their actual use-case was to let customers easily roll their own code to measure, e.g., the page-viewing behavior of only customers who'd bought more than $200 worth of stuff, which allegedly MixPanel wouldn't let you do.  Which would've been a perfectly good answer if the Entrepreneur had given it at the start of the session, instead of the whole session being about Paul and Harj trying to get at that information.

Five-second-level skill:

The 5SL skill for this problem requires:

  • Trigger:  Recognizing when your words or thoughts are too abstract.
  • Action:  Moving downward in the abstraction lattice, or moving nearer to sense input or motor output; being able to render your thoughts more specific or more concrete.

Both of these are targetable for exercises.

Pain points & Pluses:

• You want Paul Graham to believe your startup is better than MixPanel.  So you say, "My startup is better than MixPanel" - just produce the pure abstract conclusion you want Paul Graham to arrive at.  You keep trying to convince Paul Graham of this statement, saying that you have customers or that you have venture capital, but never actually move downward to the level where Paul Graham could arrive at this conclusion by model-checking.

• You want to describe what your software does, so you say it makes connections between people.  You have something specific in mind, but the words coming out of your mouth are so general that - although you're not thinking of those other cases - they could apply equally well to Facebook or telegraph lines.  Paul Graham has no idea at all what you're trying to describe and is giving you blank looks.

• The worse version - and the reason why Paul Graham doesn't just trust you, even if he thinks you're honest - is the case where you yourself want to believe your startup is better than Facebook, but you can't think of any specific thing your startup does better than Facebook, so you think of other abstract generalizations that seem to support the conclusion, like "We have smarter people" or "We got more funding earlier."  Where fuzzy thinking is motivated, overly abstract thinking is motivated.

• Abstract words can also avoid emotion.  George Orwell:  "Defenceless villages are bombarded from the air, the inhabitants driven out into the countryside, the cattle machine-gunned, the huts set on fire with incendiary bullets: this is called pacification."  Or contrast "Humanity is awful, it'd be better for the planet if we all died" to "Everyone including my little sister is awful, we'd be better off if everyone died including her."  To feel sympathy, we need enough concrete detail that our emotions can model-check the picture and be activated.

• Cognitive-behavioral therapy is the big experimentally supported version of therapy, for anyone not aware of this, bearing very little resemblance to anything Freudian.  CBT talks about using requests for specific details to interrupt thoughts looping around vague but affectively laden centers, like "I am a good husband", "I am a bad husband", or "my roommate is a slob".  How are you a good husband?  How are you a bad husband?  Which specific feature of your roommate are you objecting to?  Taboo the emotionally valent word at the center, like "slob", and replace it with something that's specific enough to be testable, or concrete enough to be acted upon.

•• Contrast also "It bothers me when you leave soda cans on the table" vs. "You're such a slob, stop being such a slob."  Or contrast:  "I'm upset" -> "I'm upset because I think the other person is looking down on me" -> "I'm upset because the person's tone of voice sounds like people who looked down on me in high school".  This is related to the incredibly important skill, search for the historical causes of your thoughts, rather than their justifications.

• Focusing on the specific details of a concrete example, instead of repeating a word or arguing about a category, can interrupt Sneaking in Connotations and Arguing By Definition.

• All the failures of concreteness warned against in the Mysterious Answers sequence, where you go on and on about how Wulky Wilkinsen is a post-utopian without ever once asking or imagining how the world ought to look, and what you yourself should experience, if that were true or alternatively false.

• Visualizing specific examples often improves quality of thought in general - we're often smarter when we're using both model-checking and deduction, visualizing a picture of what we're supposed to be reasoning about, constantly checking our deductive steps against some specific model those deductions are supposed to be true about.  Saith Richard Feynman:

I had a scheme, which I still use today when somebody is explaining something that I'm trying to understand: I keep making up examples. For instance, the mathematicians would come in with a terrific theorem, and they're all excited. As they're telling me the conditions of the theorem, I construct something which fits all the conditions. You know, you have a set (one ball) - disjoint (two halls). Then the balls turn colors, grow hairs, or whatever, in my head as they put more conditions on. Finally they state the theorem, which is some dumb thing about the ball which isn't true for my hairy green ball thing, so I say, "False!"

 If it's true, they get all excited, and I let them go on for a while. Then I point out my counterexample.

"Oh. We forgot to tell you that it's Class 2 Hausdorff homomorphic."

"Well, then," I say, "It's trivial! It's trivial!"

• Being specific helps notice and call bluffs, should you be mischievously inclined.

"Beware, demon!" he intoned hollowly.  "I am not without defenses."
"Oh yeah?  Name three."
-- Robert Asprin, Another Fine Myth

Wannabe executive:  "I will improve communications between employees and management."
Me:  "Can you give me a specific example of how you would do that?"

Known exercises for this skill:

In our previous Rationality Camps, Anna found that her attempt to teach a unit on "Being Specific" didn't seem to work.  Her central exercise was picking a category and asking people to name examples.

This isn't to say that the Camps were unsuccessful at teaching the skill.  Attendees picked it up, not from the explicit unit, but from all the instructors having to repeatedly ask the attendees to be more specific, and then having to ask them again, while being specific themselves, until the attendees picked up the rhythm by example and feedback.

Given our present teaching technology, this skill seems transmissible from master to apprentice, but not yet replicable by exercises.  That's why we're turning it over to you.

Comments (306)

Comment author: Morendil 03 April 2012 06:52:04AM *  55 points [-]

her attempt to teach a unit on "Being Specific" didn't seem to work.

How specifically did it not work?

(ETA: I should probably add I'm not being mischievous here; "doesn't work" is a trigger phrase for me, born out of extensive experience of dealing with useless bug reports. It systematically unpacks into at least two questions, "what behavior were you expecting" and "what did you get instead".)

Comment author: Vaniver 03 April 2012 05:02:41AM *  47 points [-]

An example of this that will be familiar to any programmer, and was taught to me in grade school, is "give orders to a malicious idiot." The teacher has the students write down the algorithm for a simple task, like "sharpen a pencil," with a wooden pencil and an old crank-operated sharpener as the props.

Typically, people begin with something like "stick the pencil into the sharpener, then turn the crank," which the teacher will do by ineffectually pushing the side of the pencil against the sharpener while turning the crank. The students revise to "stick the end of the pencil into the hole in the sharpener, then turn the crank," which the teacher will do by sticking the eraser into sharpener. (There are, if I remember correctly, four or five different features you can require the pencil-sharpening algorithm have, like which end of the pencil to stick into what part of the sharpener, which way to turn the crank, to hold the pencil still so it doesn't just spin with the crank or fall out if the sharpener is oriented poorly.)

(This will be familiar to programmers because going from the basic algorithm to code requires a level of detail that can't be faked.)

Comment author: AShepard 03 April 2012 04:12:08PM 21 points [-]

I was reminded of something similar by AspiringKnitter's post below. There is an event in Science Olympiad called Write It Do It. One person is given a constructed object made out of LEGO, K'Nex, or similar. They write a set of instructions for how to reproduce the object. These are then given to a teammate who hasn't seen the original object, who must use the instructions to reconstruct the original object. Seems fairly simple to adapt to a group setting - you could just split the group into two rooms and have them first write their own instructions and then try to follow the instructions of a partner in the other room.

This exercise and malicious idiot exercise differ in the "when" and "by whom". With a malicious idiot, your errors are pointed out immediately and by somebody else. When writing instructions, your errors don't come to light until your partner's object doesn't look like yours, and neither of you might notice until that point. It's important to notice a lack of specificity both in others (so they don't lead you astray) and in yourself (so you don't lead yourself astray), so it would probably be useful to do both kinds of exercises.

Comment author: TheOtherDave 03 April 2012 04:21:38PM 8 points [-]

There's a lower-overhead version of the LEGO exercise involving pen and paper: person A draws a design on a piece of paper and hands it to person B, who writes instructions for how to reproduce that shape and hands them to person C, who follows them. Then compare A's output to C's.

Naturally, this can be done in parallel with N people, all of whom start out as As and end up as Cs.

Of course, this kind of depends on A not knowing what's coming, since otherwise A just draws a circle or something.

Comment author: thomblake 03 April 2012 05:30:50PM 2 points [-]

This game is particularly fun when chained; A draws, B describes, C draws, D describes, and so on. Then you see how the shape transformed over time.

Comment author: RobinZ 03 April 2012 06:21:02PM 3 points [-]

There's actually an online game called "Doodle or Die" for playing this. It being an online game, however, there are a disgustingly large number of players who break the chain (willfully or non).

Comment author: atucker 04 April 2012 05:47:20AM 4 points [-]

We've played this at meetups a few times.

It hammered in the illusion of transparency pretty well. (Puppy Trampoline -> Drawing -> If you jump on a dog you make it stronger).

Comment author: RobinZ 04 April 2012 05:55:39PM 3 points [-]

I remember that! I think the biggest obstacle to clarity in the game is actually the rarity of artistic skill, not the vagueness of the written descriptions, though.

Comment author: thomblake 03 April 2012 06:23:41PM 1 point [-]

I've played a similar game in person - I think it was Telestrations. You get a word from a stack of cards, and try to draw that word. The next player guesses which word you were trying to draw, and the next player tries to draw that word (and so on). Fun party game.

Comment author: fiddlemath 03 April 2012 12:58:49PM 12 points [-]

Once a year, an acquaintance of mine gets his first-year programming class to tell him how to make a peanut butter and jelly sandwich. Even more knobs. :)

Comment author: handoflixue 04 April 2012 08:10:26PM 2 points [-]

The best question I ever encountered during an interview for a Technical Support position was to describe either that or tying your shoes. It's a great test of whether a prospective employee will be able to actually communicate troubleshooting concepts to the caller on the other end of the line, since obviously they can't use anything but words to do so :)

Comment author: Morendil 03 April 2012 06:26:56AM 8 points [-]
Comment author: [deleted] 03 April 2012 06:10:23AM 13 points [-]

I think this is a great idea! One addition I think would be useful is that (after a demo), have people get into small groups and take turns being the "malicious idiot" (instead of just the teacher playing this role). This will allow them to think of the issue from the OTHER side. (and be more kinetically interactive)

Comment author: wedrifid 03 April 2012 07:52:50AM *  1 point [-]

An example of this that will be familiar to any programmer, and was taught to me in grade school, is "give orders to a malicious idiot." The teacher has the students write down the algorithm for a simple task, like "sharpen a pencil," with a wooden pencil and an old crank-operated sharpener as the props.

... Then the malicious idiot stabs you in the eye with the pencil. Oh, the malicious idiot was supposed to follow orders and only follow orders? Why didn't you say so?!

Comment author: Vaniver 03 April 2012 02:10:09PM 3 points [-]

Why didn't you say so?!

Because I love setting other people up for jokes.

Comment author: Viliam_Bur 03 April 2012 08:58:15AM *  2 points [-]

I guess the malicious idiot is not suppossed to be creative, but lazy. They should use the simplest possible explanation -- only the simplicity is not measured by common sense, but by something like Solomonoff prior.

Comment author: DSimon 05 April 2012 08:35:39AM 2 points [-]

No, it's maliciousness, but very specifically aimed maliciousness. They don't want to hurt you, they just want to demonstrate that you are bad at giving directions.

Comment author: Spurlock 04 April 2012 03:09:21AM 2 points [-]

I don't think this a good restriction. Consider the fact that Hanlon's Razor is even a thing:

Never attribute to malice that which is adequately explained by stupidity.

This suggests that people often mistake stupidity for malice. So given that in these examples, your opponent probably does secretly understand what you're communicating (most of us know deep down how to sharpen a pencil), it might be necessary to have malice/creativity play the part of inferential distance. Otherwise you may learn to anticipate an unrealistically rational audience, one which never comes in with incorrect preconceived ideas, or lacks the necessary technical vocabulary, or seems to practice selective hearing, etc.

In short, original seeing is the exception, not the rule, so the opponent should be at least slightly hostile in his/her interpretations to account for this.

Comment author: CronoDAS 03 April 2012 08:15:44PM *  1 point [-]

Darn it, this was the first thing I thought of, and now I can't get any credit for it!

See also: How to cook scrambled eggs

Comment author: lincolnquirk 03 April 2012 09:26:50AM 19 points [-]

Idea: train the skill "ask for examples" instead, which seems easier to train, and bootstraps you into being more specific. I am not usually confused about my own thoughts, but I am often confused when others are trying to explain their ideas.

Example: explaining a startup idea to my friend today, I said "...and the viral strategy for this payments app could be loaning money to people". He was confused and asked for an example, so I said "let's say we are at a coffee shop. I have no cash, you have the payment app, and I want to borrow money for coffee. You tell me to download the app, so you can loan me the money. Then I can get my coffee, but to pay you back, I have to complete the signup process." -- here, I was incorrectly assuming my friend had all the same context I did with respect to viral strategies.

Exercise: after explaining the virtues of asking for examples, start to move onto another topic. A confederate in the audience yells "can you give an example?" Everyone giggles, the instructor says "I'm glad you asked!" and the exercise is explained: Students pair off and start telling stories to each other, intentionally leaving out details. The other person's job is to try to figure out where the details are being elided and say "example?" at the right time.

For the rest of the session, the instructor intentionally leaves out details, awarding points for the first student who notices missing details and asks for example.

Intentionally set the atmosphere of jumping in / cutting in on an explanation, since it is much more valuable right then as opposed to later.

Comment author: Morendil 03 April 2012 11:00:59AM *  18 points [-]

A friend of mine makes these stickers:

Comment author: palladias 03 April 2012 01:51:12PM 17 points [-]

Idea One: Monday/Tuesday Game

On Monday, your proposition is true. On Tuesday, your proposition is false. Tell me a story about each of the days so I can see how they are different. Don't just list the differences (because you're already not doing that well). Start with "I wake up" so you start concrete and move on in that vein, naming the parts of your day that are identical as well as those that are different.

Idea Two: Sabotage Game

You're definitely right, but unfortunately there's a malevolent actor who wants to make you look a fool. And even more unfortunately, he's creative and has a lot of resources. What does he sabotage? What evidence does he counterfeit? (This way you're identifying the examples or the proofs by looking at them as structurally crucial.)

Comment author: 27chaos 29 July 2015 07:25:05PM 0 points [-]

Idea Two: Sabotage Game

You're definitely right, but unfortunately there's a malevolent actor who wants to make you look a fool. And even more unfortunately, he's creative and has a lot of resources. What does he sabotage? What evidence does he counterfeit? (This way you're identifying the examples or the proofs by looking at them as structurally crucial.)

I really like this and wish I'd seen it earlier. Good idea.

Comment author: ITakeBets 03 April 2012 11:33:17PM *  16 points [-]

At the risk of escalating the Meta War, I think "be specific" and "be concrete" are themselves too general and abstract to engender good exercises. They look more like "do algebra" than "factor a polynomial". Not that you wouldn't get some interesting responses if you said, "We need ideas for teaching how to do algebra," but most of them probably wouldn't make students better at factoring a polynomial-- analogously, I like the "teach me to sharpen a pencil" game, and it would make a fun and striking activity, but I'm not sure it would help students learn to explain their business plan better in an interview. If you want students to communicate judgements and opinions better, teach them to do that.

In this case, I would unpack "specific" into two parts: concrete and relevant. To make a statement more concrete, you talk about how qualities can be measured or observed ("Yellow is a color" becomes "Yellow is the color of a dandelion" or "yellow is the color of the emission spectrum of sodium"). To make it more relevant, you relate it to a goal or higher-level question ("These scissors are dull" becomes "I can't use these scissors to cut hair").

For a straightforward activity, give students a list of statements, and have them classify them as vague, concrete, relevant, or concrete and relevant. Example:

  • Tom is too short.-- vague
  • Tom is too short to play basketball.-- relevant
  • Tom is 5'6".-- concrete
  • Tom is 4 standard deviations below the mean height of a college basketball player.-- concrete and relevant

For a higher-level activity, give students a vague statement, and ask them to make it concrete and relevant. Example:

Having a car gives me more flexibility. becomes: Having a car lets me get from my home to the southwest corner of town in 30 minutes; other transportation options would take more than an hour and a half. There are many employment opportunities in that part of town, so I have more work options with a car.

Comment author: HonoreDB 03 April 2012 03:27:26PM *  14 points [-]

Argh. I'm reminding myself that Retroactive Rewards Rage is a cognitive fallacy. Is there a formal name for it? I bet you could induce it in chimps.


Abstraction Telephone

Divide into at least 4 groups, of minimum size 1 and maximum size maybe 5. Each group gets a different short passage. They collaborate to translate the passage their choice of either "one rung up," making it all more abstract, or "one rung down," making it all more specific. Group N then passes their translated passage only, not the original, to Group N+1 modulo the number of groups. Then each group performs the same operation, then passes it to the next group in line. I think two iterations will be enough to get something entertainingly mangled, so then each group in turn performs the passage they've been handed for the audience. The remaining people may try to guess what the original passage was.

Example (2/3rds stolen from George Orwell):

Group 1 gets:

"I returned and saw under the sun, that the race is not to the swift, nor the battle to the strong, neither yet bread to the wise, nor yet riches to men of understanding, nor yet favour to men of skill; but time and chance happeneth to them all."

They note that everything but the last clause is pretty specific, so they decide to move it up the ladder. They write

Objective considerations of contemporary phenomena compel the conclusion that success or failure in competitive activities exhibits no tendency to be commensurate with innate capacity, but that a considerable element of the unpredictable must invariably be taken into account.

Group 2 gets this. They note that it's extremely abstract, so they decide to make it more specific. They write

Any competent Magic: the Gathering player will beat the best one in the world at least 10% of the time. Statistical models of baseball games show that a significant amount of variance cannot be attributed to any stable ability of players and teams. Even the most carefully hedged investment strategy can be wiped out by a black swan.

Group 3 does a dramatic reading of this passage, and maybe Group 4 tries to guess what the original passage was.


Group 2's first passage was this, from Poor Richard's Almanac:

  • The Family of Fools is ancient.

  • Necessity never made a good bargain.

  • If Pride leads the Van, Beggary brings up the Rear.

  • There’s many witty men whose brains can’t fill their bellies.

  • Weighty Questions ask for deliberate Answers.

They decide to add specificity. Each group member picks a proverb to translate, and they get:

  • The man who is amazed that whenever he looks at his clock, it reads "11:11," would in prehistory be the man who thinks he can control the weather with a ritual.
  • A thirstier person will pay more for water.
  • If you won't accept handouts from your relatives now, you might be begging from strangers later.
  • The market value for a degree in English is pretty damn low.
  • When given a difficult question, wait 5 minutes, by an actual clock, before proposing solutions.

Group 3 gets this, and decides to take it even further down the ladder into specificity, giving real-life examples of these situations, or empirical economic formulae.


Group 3 gets a printout of The eHow page on selecting a digital camera. They take it up the ladder, and write

"Higher resolution and zoom capability should be priorities. Be sure to use a good quality lens. Look for cameras with more and better features. Don't pay more than you can afford!"

Group 4 gets this. They note that they can't reverse this process, since too much information has been destroyed, but they can further abstract it to

"Figure out what you want in a camera, and how much you're willing to pay."

Comment author: Eliezer_Yudkowsky 04 April 2012 08:07:29AM 4 points [-]

We'll totally do retroactive awards for anything we try from Check Consequentialism.

Comment author: Morendil 03 April 2012 07:00:15AM 14 points [-]

Exercise - filing bug reports. Have attendees use software that sucks (that they provide, e.g. on their mobile device, or that you provide, e.g. over the Web), or recall occasions where they used software that sucked. Ask them to describe the problem they encountered in enough detail that someone else can reproduce it, or deduce something useful such as a workaround.

Comment author: fubarobfusco 03 April 2012 07:45:18AM 5 points [-]

"It needs to be about 20% cooler."

Comment author: AspiringKnitter 03 April 2012 05:45:12AM *  13 points [-]

I think you should go with Vaniver's idea. (Edit: Vaniver now has multiple ideas up. I mean the one about giving orders to malicious idiots. Completely off-topic: that's also a useful way to explain tasks to people with Asperger's Syndrome or other neurological oddities that cause executive dysfunction.)

I also think this reminds me of something (fiction) writers talk about a lot: they've hit on the way people won't sympathize with "a billion people died/starved/were tortured/experienced dust specks in their eyes" but will sympathize with "Alice was mobbed by dust specks and blinded" and will sympathize even better if you give some specific details about how it felt. And then they go on to talk about how to make Alice someone the reader cares about and how to craft sentences and other stuff that's relevant to them but irrelevant here.

But maybe something like making up a character and talking someone through xyr experience using the product step by step, in the kind of detail a novelist would use to describe the climactic fight scene.

Another idea that occurred to me is some sort of exercise where two people would pair up. One would have to do a novel task or navigate some kind of obstacle course blindfolded and the other would have to give directions. They wouldn't be able to get away with "turn right at the statue" but would instead have to give directions like "turn right at the big smooth stone thing" and... I guess if you were doing something like that, you'd want to give the non-blindfolded partner a picture or map and NOT let them see the one doing the actual task. Otherwise they'd just be able to say "okay, turn right now... turn left... turn left again..." and that would defeat the purpose.

Comment author: Oscar_Cunningham 03 April 2012 09:30:01AM 4 points [-]

Upvoted for that last paragraph.

Comment author: dbaupp 04 April 2012 08:40:43AM *  2 points [-]

(Summary: Orienteering with the navigation and movement separate. This exercise requires both people to be specific to not get lost, and it can be extended by adding in a race aspect (trying to be specific under pressure!).)

The last paragraph made me think of "MoboGoGlobo", which is an orienteering event where there are two participants: one doing the navigation who guides the other via phone.

(I'm not sure how familiar with orienteering many people are, so I'll give a quick intro) When participating in an orienteering event one normally has to visit a series of markers in a predefined order as quickly as possible (or not, if one isn't feeling like racing, entirely up to the participant). One has a detailed map (although, importantly, it rarely has street names) that indicates the location of these and the order to visit them in, e.g. the two maps here (the pink (or purple) triangle is the start location and the numbered circles are the markers).

When competing, one uses all sorts of clues to make sure one is going in the right direction and one isn't lost (and to become unlost), like most obviously the shape of nearby buildings, the topography (e.g. a steep hill), a fork in a track or stream, or more subtly things, like a bend in a track or the position of a power line on the next hill. (Conventionally, one also has a compass, which one uses to orient oneself correctly.)

In this exercise you need two people: one with the map ("navigator"), the other actually on the territory that corresponds to the map ("runner", this doesn't imply that one needs to run though): it requires the navigator to describe exactly where to go ("go past the building" < "go to the left of the building that has a round canopy outside"), and also for the runner to describe what they see so that the navigator can keep track ("There is a line of trees" < "A line of trees starts just to my left and goes directly away").

There are multiple levels of specificity too: the best teams will have a navigator who gives a good general overview of the location of the marker but also detailed instructions along the way, and the runner who gives compact but detailed progress updates (There is limited bandwidth too).

I'm sure many people won't have done any orienteering before, so this would have to be kept simple. Even just a reasonably large building and using its floor plan as a map, maybe adding some details, and removing any names of rooms (preferably neither participant would have been in the building before). (Other possible locations include parks and university campuses (these can be very complicated though!))

Doing it as a competition adds an element of pressure which is possibly too much for people who are unpracticed at the skill (and at reading a map), but it would be an easy way to extend the exercise and test the skill more.

Comment author: erratio 03 April 2012 12:20:53PM 2 points [-]

Yeah, seconding the "blind obstacle course" exercise, although you don't usually need to make it more difficult by not letting the person giving instructions watch (mostly because it's difficult just to walk in a straight line without visual cues, let alone execute precise turns). It's a common leadership/working as a group game and people usually need to watch other attempts go wrong 3 or 4 times before hitting a useful level of specificity.

Comment author: Vaniver 03 April 2012 02:20:04PM 1 point [-]

I typically use a permalink to refer to comments that aren't upthread. (Thanks for the recommendation, by the way!)

Comment author: [deleted] 03 April 2012 07:18:38AM *  12 points [-]

Also, in my quest to make these all relatable to improv games:

More Specific (it's not a very good video, sorry), is a game in which two people act out a scene, with the audience (or moderator) occasionally demanding that they be "More Specific"


B: Good job finishing your paper on time!
C: Yeah, I had the hardest time finding my references...
Audience: More specific!
C: My kid thought that my reference books would be better with all the pages ripped out, and I couldn't find the pages with my quotes

I'm thinking that I actually might have seen this game mentioned on LW before, but I didn't a quick search and didn't see it.

Spitball (not a vid) is a game in which "Players take a mundane object suggested by the audience and elaborately detail it", generally using "Yes, and..." technique.

An improv-y style warm-up game that I just thought up, upon which I bestow the name "Something". In this game, everyone gets in a circle. The starting person (or moderator) comes up with a sentence that has a lot of "somethings" and "things" in it. Each person makes it more and more specific.

1: Did you something the thing with that thing that somethinged?
2: Did you see the thing with that, etc....
3: Did you see the bug with....
4: Did you see the ant with....
5: Did you examine the ant....

As with all of these games, the "goal" is to keep it quick.

In beginner level, it goes around the circle clockwise, and the sentences are kept short.

In advanced level, after you go, you shoot it off (the clap-point motion) to a random person, and the sentences can be longer.

I have no clue how well this game actually works (having just thought of it), if anyone ever tries it out, let me know!

Comment author: lincolnquirk 03 April 2012 08:40:15AM *  10 points [-]

Exercise: incentivize both teacher and student participants in Vaniver's "malicious idiot" exercise. Give the student points when she is successfully more specific, and give the teacher points when he finds a new way to misinterpret the student instructions.

Example: how to brush your teeth?

S: hold your toothbrush

T: (picks up toothbrush with teeth) (1 point)

S : hold your toothbrush between your thumb and fingers of your right hand (1 point)

T: (makes a fist, puts toothbrush on outside of fingers) (1 point)

S: argh. Like this! (picks it up as example) (2 points for being concrete)

T: (mimics perfectly)

S: great, now... Brush your teeth. (laughter)

I could go on, but hopefully you get the point.

Comment author: Mass_Driver 03 April 2012 09:50:43PM *  3 points [-]

I think this is the best exercise posted so far. Unlike the other exercises, it can be explained to ordinary middle schoolers in 90 seconds, it engages both visual and tactile senses, it is competitive, it is devoid of perverse incentives, and it will usually be fun to play and funny to watch.

Comment author: Armok_GoB 04 April 2012 06:18:31PM 4 points [-]

devoid of perverse incentives

Not so, you're better of improving in small increments than doing the best you can immediately.

Comment author: Dues 12 July 2014 03:36:07AM 0 points [-]

That depends on the scoring system. If the judge grade exponentially for better answers, then small increments are a loosing choice.

Comment author: Morendil 03 April 2012 07:10:12AM 10 points [-]

Exercise - fetch something from the kitchen. This happens all the time at home and drives me crazy. "Get me the dentist's papers, they're in the drawer." What drawer, in what room, what do the papers look like, are they in some sort of container... Reproducing this in a residential training setting opens up interesting possibilities. For instance, while giving attendees a tour or taking them from place to place tell them (separately) to pay attention to a specific item ("this is your target for the exercise after lunch") in a non-central location. The next day, pair people up, have them describe the target to each other, fetch targets, debrief.

Comment author: Username 05 April 2012 06:26:51PM *  9 points [-]

Unrelated to the post, but I'm not sure where else to suggest rationality exercises. So I'd like to revive an idea I saw here a while back called 'What Did You See?' (I can't take any prize if it's selected because it's not mine). I think it would be a wonderful game for developing curiosity and noticing specifics. But above all its purpose is learning that you can learn, which I think even in the rationality community is an important lesson that helps to reignite the inquisitive spark.

At home there was a game that all the parents played with their children. It was called, What Did You See? Mara was about Dann’s age when she was first called into her father’s room one evening, where he sat in his big carved and coloured chair. He said to her, ‘And now we are going to play a game. What was the thing you liked best today?’

At first she chattered: ‘I played with my cousin . . . I was out with Shera in the garden . . . I made a stone house.’ And then he had said, ‘Tell me about the house.’ And she said, ‘I made a house of the stones that come from the river bed.’ And he said, ‘Now tell me about the stones.’ And she said, ‘They were mostly smooth stones, but some were sharp and had different shapes.’ ‘Tell me what the stones looked like, what colour they were, what did they feel like.’

And by the time the game ended she knew why some stones were smooth and some sharp and why they were different colours, some cracked, some so small they were almost sand. She knew how rivers rolled stones along and how some of them came from far away. She knew that the river had once been twice as wide as it was now. There seemed no end to what she knew, and yet her father had not told her much, but kept asking questions so she found the answers in herself. Like, ‘Why do you think some stones are smooth and round and some still sharp?’ And she thought and replied, ‘Some have been in the water a long time, rubbing against other stones, and some have only just been broken off bigger stones.’ Every evening, either her father or her mother called her in for What Did You See? She loved it. During the day, playing outside or with her toys, alone or with other children, she found herself thinking, Now notice what you are doing, so you can tell them tonight what you saw.

She had thought that the game did not change; but then one evening she was there when her little brother was first asked, What Did You See? and she knew just how much the game had changed for her. Because now it was not just What Did You See? but: What were you thinking? What made you think that? Are you sure that thought is true?

When she became seven, not long ago, and it was time for school, she was in a room with about twenty children – all from her family or from the Big Family – and the teacher, her mother’s sister, said, ‘And now the game: What Did You See?’

Most of the children had played the game since they were tiny; but some had not, and they were pitied by the ones that had, for they did not notice much and were often silent when the others said, ‘I saw . . .’, whatever it was. Mara was at first upset that this game played with so many at once was simpler, more babyish, than when she was with her parents. It was like going right back to the earliest stages of the game: ‘What did you see?’ ‘I saw a bird.’ ‘What kind of a bird?’ ‘It was black and white and had a yellow beak.’ ‘What shape of beak? Why do you think the beak is shaped like that?’

Then she saw what she was supposed to be understanding: Why did one child see this and the other that? Why did it sometimes need several children to see everything about a stone or a bird or a person?

  • Doris Lessing, "Mara and Dann"

- RichardKennaway, February 2011 Quotes Thread

Comment author: atorm 08 April 2012 01:30:33AM 0 points [-]

I realized that the movie was "It's a Wonderful Life" within the first paragraph. Consider adjusting your estimates of readers. Also, an imdb review that gave me the plot of a movie would not be a good review. It would be a synopsis. That review told me that the acting was good and the story heartwarming. I don't think that it is a good subject for criticism of specificity.

Comment author: Username 08 April 2012 08:42:00PM 0 points [-]

Are you replying to the right comment?

Comment author: Benquo 03 April 2012 02:01:53PM *  8 points [-]

Several comments mention guessing games. Here's my variant:

One "communicator", one or many askers.

The communicator starts by describing something in the most general terms they can think of.

Each round, the asker(s) can either ask a question, or guess. The question has to be about an attribute of the thing, not its name, and they can't say "Is it X?" or "What is it?". The communicator answers, vaguely if the question is vague, getting specific if they're backed into a corner.

When they guess, the game is over. If they guess wrong, they lose. Then you try to figure out what your next question should have been.

If they guess right, they win and get congratulated.

"It's a way of moving things from place to place."

"What kinds of things?"

"Oh, things you want moved."

"How big can these things get?"

"No more than six feet high, a couple feet wide, less than a foot long."

"Are they inert and durable?"


"Are they fragile?"

"Depends on your standards."

"Would they break if you dropped them from a foot?"


"From ten feet?"


"What materials are they made from?


Eventually the guesser figures out that you're moving people, one at a time, with no equipment except the person's own legs.

"I know! Walking"

"WRONG - you forgot to ask if the person's feet are ever both off the ground, and whether their posture is upright or at an angle. Could have been leaping or running."

Comment author: Incorrect 06 April 2012 09:19:15PM *  0 points [-]

The answerer is being cooperative here though. This is how I would have answered:

"It's a way of doing stuff to stuff"

"What kinds of stuff?"

"Stuff that's like some stuff but unlike other stuff"

"How big is the stuff?"

"Around the size of other stuff"

"Is the stuff inert and durable?"

"As much as that sort of stuff can be"


Comment author: Benquo 06 April 2012 09:35:30PM 0 points [-]

Yes, but sufficiently specific questions should be anle to corner any uncooperative answerer playing in good faith.

You don't have to accept that answer to "how big," for example. You can ask, is the stuff organized into discrete items? If so, are they bigger than a cubic mile? If not, are they smaller than a cubic foot? And so on.

Comment author: faul_sname 06 April 2012 08:58:04PM 0 points [-]

Very strongly reminds me of this as well. I wonder what sort of exercise might combine the two.

Comment author: BlueSun 03 April 2012 06:13:50PM 7 points [-]

Company mission statements are notoriously abstract and might make a good starting place. If someone didn't know anything about a company and they went and read the mission statement, they probably wouldn't have a much better idea of what the company actually did.

For example, if (stereotypical) Grandpa asked you what Google was and you replied, "they organize the world’s information and make it universally accessible and useful" you probably wouldn't do much to help him understand what Google is (despite that being one of the best mission statements I can think of). Instead, if you gave a specific example such as: "If you're driving to a new store you can type in the store address and Google will print out a map of how to get there, along with detailed instructions. It's more convenient than a traditional printed map because if you don't know the address you can type in the store name and Google will tell you the address and show you pictures of the view from the street so you'll be able to recognize it when you're driving there." Grandpa would probably have a better idea of what Google does.

So the activity would be to take a company mission statement (abstract) and come up with several examples of specific things that the company did that you could use to describe the company to your grandparents if they'd never heard of it before. The reason to start with the mission statement would be so that participants would be able to mentally contrast abstract statements (that wouldn't help Grandma understand) with specific examples (that would help) and so hopefully learn to avoid making the abstract statements themselves. (For those participants who are Grandparents, they can use companies and products that no longer exist that younger people don't understand and pretend they're explaining it to their grand kids )

Comment author: [deleted] 03 April 2012 04:26:41PM *  7 points [-]

One way sub-skill of being specific is learning to focus on details, instead of big picture.

For example, if a manager is vague they focus on the big picture and tell their employee "You need to stop arriving late."
If a manager is specific, they focus on details and tell their employee: "In the past week, you have been more than 15 minutes late two times. If you are running more than 10 minutes late you have to call. If you are more than 15 minutes late 3 times in a month, you will get written up, etc."

I think a fun way to teach how to focus on details is to do a quick practice of cold reading.

First, you can look at the big picture (generalities), by looking at a persons general demographic, "middle aged female in a suit," and list what you can guess about them, and how certain you are of that.

Next, look at the details. Since everyone will be new to this, give them a list of specific things to look for. If you are using pictures of people to read, or are using the instructors as examples, you can make sure that the list is tailored for your examples.

Example list: Wedding band, frays on clothes (especially on heels of jeans), hair and nail style (how much they care about looks), footwear (how practical? how expensive? how comfortable?), phone (watches can work too, but it's harder to tell an expensive/cheap watch from an expensive/cheap phone), bearing/expression

Have them make a list of all the details that they can find about the subject. Next, have them update their guesses about that person, and their certainties. Discuss how the specific details can lead to either more specific conclusions, or more certainty.

"Well, at first I thought she was a business person, because she was wearing a suit. But then, when we looked at the specifics, I noticed that her phone and shoes were cheap, and she looked nervous. Now I think she is going to an interview or something.

I don't know much about cold reading, but I have a friend who is a magician/"psychic", and I can ask him for any interesting things to look for, if there's interest.

I think people will remember this because a) it is fun, and b) it is a useful skill in itself .

Edit: You can also pair them up or put them in groups and have them cold read each other, but people would likely be uncomfortable with that (both possibly stating negative implications about a person (i.e. "well your hair is greasy, so...") and being the subject of such scrutiny. But it could possibly work at a meetup where people already know each other, and are already a bit more rational. (People can declare Crocker's Rules if they want.)

Edit 2: If you don't want to look at people this way (ethical reasons, or just think it will make people uncomfortable), you can instead call the game "Be Sherlock", and examine a scene the same way. (i.e. "Oh, there's a lit cigarette, so someone was here recently. The TV is playing Sesame Street, so there was probably a kid here too. It's an educational show, so the adult probably cares about the kid(s), etc.)

Comment author: lincolnquirk 03 April 2012 09:08:29AM 6 points [-]

Exercise: Stay in the entrepreneurship domain and channel PG. Pretty much everyone has startup ideas, right? Actually apply PG's algorithm to the participants' real startup ideas. 

(I think his algorithm is something like: do I understand the core idea? Is it something people want? How do you know? Is it something people will pay for? How do you know? What are the obvious flaws? Why are you the team to do this? Why is now a good time for this? Is it working? How do you know? What insights/surprises?)

Example: I love Magic the Gathering and I want to have a place for people to talk about how awesome the decks are--

T: be specific!

A site to talk Magic strategy. (+1 conciseness)

T: how do you know people want this?

Well, people love magic and talk about it all the time.

T: I'm not convinced.

Well, I talked to a famous magic player (+1 specificity) and he told me this story about how he invented this deck back in the day on the starcity forums, and he couldn't have done it without the community [lincoln note: this is totally made up], so we want to enable that to happen more. (too general, but the teacher doesn't press yet, instead noting the obvious flaws for later discussion)

T: and people will pay for this?

Yes, magic is an xxx million dollar industry

T: (look of disapproval)

Tournament players care deeply about having & knowing about the best "tech", and the scene changes rapidly every 3 months as new cards are printed (+0.5 specificity)

For example, if I am trying to decide whether to play a mostly red deck, I want badly to know the field - if lots of people are packing pro-red hate, I might not want to play red. (+1)

T: (returning to obvious flaw) why will people post their secret tech? Isn't it advantageous to keep it hidden?

Ah, we will solve this with culture. It's very open environment. (-1)

T: huh?

We found that people are willing to post their decks.

T: how?

Well, we put the forums up and people posted decks! (+1 for specificity. But minus for this actually being a good idea.)

Cutting off my example now because it is getting long.

Comment author: majus 11 April 2012 06:14:26PM *  5 points [-]

Why is "be specific" a hard skill to teach?

I think it is because being specific is not really the problem, and by labeling it as such we force ourselves into a dead-end which does not contain a solution to the real problem. The real problem is achieving communication. By 'achieving communication', I mean that concepts in one mind are reproduced with good fidelity in another. By good fidelity, I mean that 90% (arbitrary threshold) of assertions based on my model will be confirmed as true by yours.

There are many different ways that the fidelity can be low between my model and yours:

  • specific vs abstract

  • mismatched entity-relationship semantic models

  • ambiguous words

  • vague concepts

Surely there are many more.

Examples of what I mean by these few:

  • specific vs abstract: dog vs toy chihuahua puppy

  • model mismatch: A contract lawyer, a reservoir modeler, and a mud-logger are trying to share the concept "well". Their models of what a "well" is have some attributes with similar names, but different meanings and uses, like "name" or "location". To the mud-logger, a well is a stack of physical measurements of the drilling mud sampled at different drilling depths. To the lawyer, a well is a feature of land-use contracts, service contracts, etc.

Another kind of model mismatch: I think of two entities as having a "has-a" relationship. A house "has" 0 or 1 garages (detached). But you think of the same two entities using a mixin pattern: a house can have or not have garage attributes (not detached). "I put my car in the house" makes no sense to me, because a car goes in a garage but not in a house, but might make sense to you for a house with a built-in garage. We may go a long time before figuring out that my "house" isn't precisely the same as yours.

  • ambiguity: I'm a cowboy, you're an artist. We are trying to share the concept "draw". We can't because the concept doesn't equate.

  • vagueness: I say my decision theory "one-boxes". You have no idea what that means, but you create a place-holder for it in your model. So on some level you feel like you understand, but if you drill down, you can get to a point where something important is not defined well enough to use.

It is difficult to know when something that is transparent to you is being misrepresented in my head based on how you explain it to me. "I know you think you understand what you thought I said, but I'm not sure you're aware that what I said was not what I meant."

I suggest an exercise/game to train someone to detect and avoid these pitfalls: combine malicious misunderstanding (you tell me to stick the pencil in the sharpener and I insert the eraser end) and fidelity checking.

  1. You make an assertion about your model.

  2. I generate a challenge that is in logical agreement with your assertionss, but which I expect will fail to match your actual model. If I succeed, I get a point.

Repeat, until I am unable to create a successful challenge.

The longer it takes you to create an airtight set of assertions, the more I get.

Then we switch roles.

So I am looking for all the ways your model might be ill-defined, and all the ways your description might be ambiguous or overly abstract. You are trying to cement all of those gaps as parsimoniously as possible.

I've left the hardest part for last: the players need to be supplied with a metaphoric tinkertoy set of model parts. The parts need to support all of the kinds of fidelity-failure we can think of. And the set should be exensible, for when we think of more.

Comment author: Vaniver 11 April 2012 08:43:40PM 1 point [-]

I suggest an exercise/game to train someone to detect and avoid these pitfalls: combine malicious misunderstanding (you tell me to stick the pencil in the sharpener and I insert the eraser end) and fidelity checking.

I suspect the Socratic method (the old one, not the bland one) fits under this heading- "put forth a proposition, and I'll demolish you with your own statements."

Comment author: thomblake 11 April 2012 08:24:39PM 1 point [-]

Sadly, "Communicate well" isn't quite as simple of a skill.

Comment author: pnrjulius 04 April 2012 08:29:38PM 5 points [-]

I'm not convinced enough that being specific IS a good thing to do, frankly. Maybe there's something else buried within the concept that we're called "specific" that really is valuable... but when I think about "being specific" and applying this to all the various cases, it very quickly leads me to doing things that are completely useless. It collapses, basically, into casuistry. At maximum specificity, I have a list of observed cases and no way of linking them together to make sense out of them.

"What is red?" becomes "Well, this thing I'm pointing at is red. Care to guess what I mean by that?" "How should an economy be run?" becomes "Well, Singapore's GDP growth in 2009 was such-and-such, while Switzerland's was such-and-such."

It seems to me that in many circumstances, we really aren't asking specific questions---we want to know the really general answer. If you're trying to derive a theory of gravitation, a long table of things that were observed to fall from various heights at various speeds may or may not help you in the process---but it certainly is NOT a theory of gravitation.

As Darwin put it: "About thirty years ago there was much talk that geologists ought only to observe and not theorise; and I well remember some one saying that at this rate a man might as well go into a gravel-pit and count the pebbles and describe the colours. How odd it is that anyone should not see that all observation must be for or against some view if it is to be of any service!"

I think the mistake that the entrepreneurs were making was not a lack of specificity; it was a failure to convey information at all. "It will connect people" is the sort of trivial normal statement that you make when you're a political candidate who has no particular ideas. (It fails the reversal test: "My product will not in any way connect people ever.") And I guess there is a certain sort of "specificity" at work here: you have to be specific enough that your assertion has content.

So for instance this is a theory of gravitation (Newton's): "All objects attract each other with a force proportional to the product of their masses and inversely proportional to the square of the distance between them." This is not: "When I dropped a rock yesterday, it hit the ground in 0.6 seconds from a height of 1.7 meters." But nor is this: "All things fall toward each other."

One of these is not specific enough; but the other is TOO specific. Hence, "Be Specific" isn't really the right advice here.

Comment author: handoflixue 05 April 2012 09:02:35PM 8 points [-]

Perhaps the better advice would be "be AWARE of how specific you are being, and in which direction you should move to facilitate communication" - sometimes you're giving a bunch of overly-specific examples without tying them together, and sometimes you're being overly general without giving someone some concrete examples to test their understanding.

My roommate and I often sit on the couch and discuss ideas for our shared D&D campaign. We exercise this skill a lot - two very common questions are: "can you give me some specific examples?" [be more specific] and "I don't get what ties these examples together" [what's the big picture / be more general]

Possibly an exercise that involves going through a quick dialog, and the audience calling out "be more specific" / "what's the big picture?" as appropriate at each step (or holding up colored flags, etc.). Another very simple exercise here, but it helps the audience calibrate to the idea, engages them a bit.

I don't think breaking people in to pair would be beneficial for this, unless you want to focus on "seeing where you are on the ladder, and WHICH direction you need to move." You seem to want to focus just on "moving down, being more specific", which seems fine to me. This just helps them to see the ladder itself, and realize that it moves BOTH ways.

You can then transition by saying "Okay, now we're going to focus specifically on techniques for moving DOWN the ladder, for being more SPECIFIC..."

Comment author: tiki1925 04 April 2012 10:56:15AM 5 points [-]

Here's a possible exercise. It's about making a set of specifications that meet a pre-defined level of abstraction, not about moving between levels of abstraction.

Toy Designer

"When I was just a wee little lad full of hope and joy My father homeward came one night and gave to me a toy A wonder to behold it was with many colours bright And the moment I laid eyes on it it became my hearts delight It went zip when it moved and bop when it stopped and whirr when it stood still I never knew just what it was and I guess I never will."

You are a toy designer for a large educational products company. You need to come up with the latest offering in the 0-3-year-old range. Your team plans to release an new version of their Cube(TM), a six-sided shape with different bits on each side that do different things. For example, the previous Cube included a handle that when turned made a clown pop up, a button that made two other buttons appear and a squishy gel pad that changed colour when warmed by fingers. You need to specify what is on each side, what a child will need to do to that side to get a reaction, and what the reaction will be. You don't need to define the mechanism or how it will work. Firstly, you have engineers for that, who will make exactly what you want as long as you tell them exactly what you want. Secondly, it won't get your design chosen over those of the other designers (which is worth a substantial bonus). The choosing panel consists of your team's manager's manager, a couple of people from Marketing, and a small number of "typical" mums and housewives chosen for their combined decades of experience in childraising and their ability to "like" the company's updates on Facebook. Previous design runs have shown you that the panel typically tends to appreciate descriptions that are similar to the way a child of 0-3 years plays with an item - literal, clear, straight-forward, and without a lot of extra context or jargon. You also know that your manager's manager doesn't tolerate vague phrases such as "provides educational opportunities and sensory stimulation" because they don't tell him how the design is going to be more interesting than your major competitor's summer release special (or even just a stick), and that's what he wants to know.

Variation/extension 1 of this exercise is to redescribe the object to include the principle of interactivity: which is that when a child changes what they do, the Cube must also change how it responds. For instance, turning a handle faster makes a higher pitched sound and turning it slower makes the sound lower pitched. The new description needs to include how each use of the Cube can be varied and what the variable response will be.

Variation/extension 2 of this exercise is to describe a multiple-step series of operations that might include more than one face. For example, from the song: "Right on the bottom were two big buttons that looked like big black eyes / I first pushed one, and then the other / and then I lifted its lid / and when I put it down again this is what it did...".

Comment author: [deleted] 03 April 2012 06:39:23AM *  5 points [-]

In psychology, the term "high-level construal" or "abstract construal" means thinking in supercategories (mammals), and the term "low-level construal" or "concrete construal" means thinking in subcategories (poodles). Being in a high level construal helps you focus on goals, and being in a low level construal helps you focus on methods and specifics, IIRC.

When testing construal effects, one way psychologists induce low-level construals is to ask an iterative set of "How" questions.

For example:
Subject- I want to get fit!
Psychologist- How can you get fit?
S- I will eat healthy
P- How will you eat healthy?
S- I will stop eating unhealthy foods.
P-How will you stop eating unhealthy foods?
S- I won't buy them in the grocery store.

These questions do NOT have to relate to the subject you are trying to be specific about! (for example, you could run an "Eat healthy" set, like above, in order to focus on specifics of a programming problem.) The question set is designed to INDUCE a thought method that focuses on specifics, not to solve a problem itself. I don't know how long an induced construal level lasts, but I would say just run through this exercise immediately before a time when you are going to have to be specific about something.

To utilize this as an exercise, people can team up and take turns being the Psychologist and the Subject. To utilize it in real life, they will have to play both roles (i.e. Ask YOURSELF "How?")

This particular exercise probably isn't the most exciting, but there's a lot of research out there, if someone wants to mine it. Off the top of my head, Kentaro Fujita does a lot of work on construal levels. I'll go through the effort of linking papers, but only if people are interested.

Comment author: Mark_Eichenlaub 06 April 2012 02:38:13PM *  4 points [-]

Exercise: What Was That All About?

Players get samples of writing from various internet sources - randomly chosen movie reviews from IMDB, news stories from Huffington Post, blog posts from Wordpress, Wikipedia articles, etc.

Player A gets to block out 5% of the words in the sample. Player B then tries to guess the topic the sample discusses.

For example, here's a semi-randomly chosen IMDB review - the first one I grabbed off the site. It got 137 "helpful" votes out of 161 voters, so it's perceived as a good review. It's of a famous movie. I've blocked 5% of the words. Try to guess what movie.

This film has become a --------- tradition in my family. We watch it every year and never tire of it. -------- is a master of creating films with a message that reinforce strong values. This is probably his greatest film in that regard. Both he and ------ have publicly stated that this is their favorite film.

The message in this film is one of courage and sacrifice for the greater good as ------, a man with big ideas about seeing the world, continually forsakes his own desires to do what is right for the ------. The second message is that each life important. No matter how insignificant we feel we are, we are all inextricably linked to each other and play an important part in the fabric of one another's lives.

------'s direction is brilliant. His genius is bringing human stories to life in a ways that not only make a point, but that totally involve the audience in the lives of the characters. He is always extremely optimistic about the human condition. He is known for testing his characters with overwhelming adversity to make them struggle to triumph in a way that causes the world to change and the character to grow. For this reason his films were always crowd pleasers and this film was the best of all in that regard.

Led by ------'s understanding hand, the actors all did a magnificent job. ------'s wide-eyed enthusiasm and boyish charm, coupled with an unbending strength of character made him the perfect folk hero. ------ was lovely and charming and attained the right balance between being supportive and inspirational. The romantic chemistry between her and ------ was subtle and charming. ------ was towering as the greedy old skinflint who was trying to take over the ------. ------ plays one of my favorite characters, as the bumbling ------ in probably his most memorable role.

This film is number ------ on AFI's list of best films of the century. It was nominated for ------ academy awards and won ------. It was swept in ------ by ------, a great film that won ------ Oscars that year but in my opinion was the lesser film. History has corrected that minor injustice by rendering ------ an enduring classic that is viewed and loved by generation after generation. Of course, I rated it a 10/10. I can't wait to see it again this ------.

Clearly, this review fails to be specific. I've pretty much just blocked all the proper nouns - names of actors, years, etc. Still, I am willing to guess that not many people will know what the movie is. (I also blocked the two words describing a plot element and the setting.)

By contrast, I went to Wikipedia and hit "random article" repeatedly until I got an article whose title was something I had heard of before. The entire article is a little long, so I took the first two segments from the beginning of the article. Try to guess the topic of the article.

------ is a ------ ------ ------ ------ that was an international success during the 1990s and early 2000s with shows being filmed in America, Finland, The United Kingdom, Australia, South Africa, Sweden, Nigeria and Denmark. Russia,Germany, The Bahamas and Japan would also compete in international shows during the series. After a lengthy break, ------ was revived in 2008 in the UK, the US, Australia and Saudi Arabia.

The concept of the show is that athletic members of the public ------ against the show's own ----- (often semi-professional or ex-athletes) to claim points in several events that require speed, strength and skill. In the final event of the show, ------ the contenders ------ against each other (with starting times based on previous events), with the ------ ------ ------ winning the episode and moving onto the next round.

A children's derivative of the concept was also made in the US, called ------ (1994–1996). A UK variant of this was aired starting in 1995, called ------.


[edit]1990s success

The initial concept for the show by Dan Carr and John C. Ferraro was held in Erie, Pennsylvania, in the USA before being sold to Samuel Goldwyn Productions/MGM where the format was adapted and televised as ------ with the first series airing over 1989-1990. As the show progressed, new events were introduced along with new ------, sometimes retiring previous ------.

Following the success of ------, other countries began to produce their own versions of the show with the UK and Finland starting production in 1992. ------ had already picked up a cult following in the UK after being shown on late night TV. The UK, most noticeably adapted the concept into a large arena (the National Indoor Arena in Birmingham), glamorizing the show, often adapting events from the American series as well as introducing many of their own, often more high-tech. Winners from the UK and Finnish series would then go over to America, to film a special show of ------ in which they competed against the current American champions along with selected athletes from other territories such as Japan and the Bahamas and South Korea.

In early 1995, the first full scale international competition was launched in which selected ------ from the American, Finnish and British series competed against contender champions from those three countries. A fourth country, Russia was added but as they did not have their own domestic series, the ------ and contenders were hand-picked by Russian TV producers. The Finnish series ceased production after ------.

In 1995, Australia began production of their own show, basing it on the UK series. After the first series, a three part 'Ashes' mini series was filmed in Australia, in which a selection of British and Australian ------ faced champions from the opposing countries. Australia then went on to compete in ------ along with the UK and America. Russia also returned, even though they still did not have a domestic series. Germany and South Africa also competed even though they too did not have their own domestic series.

This is definitely more specific, but it still could be significantly better. It's probably clear that it's a television show where people have some sort of athletic competition, but it's not really clear what sort of athletic competition it was, what the general mood of the show was like, etc. There is a bit more information on this later in the article, but not quite as much as one might expect. Still, you know a lot more about this show than you know about the movie. This is despite the blanks blocking significant material; I blocked out verbs like "compete" and "race". In the movie review, there wasn't even such material available to block.

The answers are It's a Wonderful Life and Gladiators television franchise.

Passages that aren't specific could describe many different things, so by blocking out a few words, the original topic is lost. Highly-specific passages don't have this problem. This exercises teaches how to be on the lookout for specific and non-specific writing, but also gives us some data on what sorts of writing tend to be specific and what sorts do not. That way you know when to be especially aware of the "Be Specific" skill. (I predicted ahead of time that artistic reviews would be very non-specific and Wikipedia articles would be very specific.)


A representative from team A writes a passage. Team B blocks 5% of the words. The rest of team A tries to fill the words back in. The number of words correctly filled in is the team's score.

Rewrite non-specific passages to be specific.

Write your own content in the same style, but with a list of taboo words. For example, an article about the Gladiators series that taboos "gladiator", "television series", "competition", etc.

All players use a particular source of content and search through to find the longest contiguous passage they can that doesn't reveal the topic. For example, one might through IMDB reviews until you find a passage that's 200 words long and doesn't let you identify the movie, even with no words blocked.

Comment author: atorm 09 April 2012 12:33:43AM 3 points [-]

I realized that the movie was "It's a Wonderful Life" within the first paragraph. Consider adjusting your estimates of readers. Also, an imdb review that gave me the plot of a movie would not be a good review. It would be a synopsis. That review told me that the acting was good and the story heartwarming. I don't think that it is a good subject for criticism of specificity.

Comment author: Mark_Eichenlaub 09 April 2012 11:50:35AM *  3 points [-]

Thanks for letting me know you found it out so quickly.

By specificity for the review, I didn't mean that it should summarize the plot. Instead, when some general statement is made, there should be some connection to the movie that supports it. Jimmy Stewart has boyish charm? When? What scenes? What about them?

Contrast to Roger Ebert's review. An excerpt:

Even the corniest scenes in the movie--those galaxies that wink while the heavens consult on George's fate--work because they are so disarmingly simple. A more sophisticated approach might have seemed labored.

This is a specific example supporting his statement at the beginning of the paragraph that, ""It's a Wonderful Life" is not just a heart-warming "message picture.""

Comment author: KateGladstone 04 April 2012 05:18:27PM 4 points [-]

A further exercise (maybe especially for children/beginners) — each student is to make up a new word. The made-up word must refer to something (real or imaginary) that does not have any specific name in the English language (or in whatever language is the working language of the group. (Examples of possible new words: "Flonk — that portion of the back of the human hand which does not contain any of the fingers" "Wedlaw — a term for the kinship relation between two people whose former spouses have married each other" "Spup — an imaginary creature with the head of a snake and the body of a puppy.") Each student then gets 30 to 60 seconds to teach the rest of the class his/her new word (and its meaning) without ever verbally defining the new word. (For example: you can draw a "spup" on the blackboard, you can outline/caress the "flonk" of one hand with the index-finger of your other hand, or you can state that two particular people in the group are "wedlaws.") After explaining the new word in this way, each student then asks the group /a/ to provide/state/draw examples of who/what would NOT be a "spup" (or a "wedlaw" or a "flonk" or whatever) and /b/ to state in each case "What indicates that this drawing is NOT of a spup?" (or "What shows that what you're pointing to isn't a flonk?" or "How do I know that my mother is NOT the wedlaw of any of her in-laws or other identifiable relatives?" After each such "vocabulary lesson," each student then gets 2 to 5 minutes to ask the rest of the class to define the word they have just learned. The "vocabulary teacher"-of-the-moment then grades each answer for accuracy and asks what could have been done to prevent error ŵithout ever actually giving a verbal definition of the word.

Comment author: jimrandomh 04 April 2012 03:23:01PM *  4 points [-]

This exercise is not for being specific, but just a general rationality-skill exercise that I think is useful.

Trivial Deduction

In every conversation, we hear hundreds of statements. Each of these implies many others - some directly, through definitions and linguistic rules that border on the tautological, some in combination with background knowledge, and some indirectly through multi-step inferences. Because the implications of each statement are too numerous to handle, we apply a strong filter to what reaches our attention: a statement reached by inference must be interesting, surprising, or connected to another interesting statement that was previously isolated.

The goal of this game is to turn off that filter, temporarily, and to pay attention to available deductive steps that would normally be too trivial to mention. You will be given a simple statement of five words or less, such as "the door is open". For two minutes, write down single-step deductions, also five words or less. For example

There is a door. The door is passable. The door was opened. People can enter. People can exit. The door isn't closed. There is a doorway. There is a wall. Air can circulate. Temperature will equalize. The door can open.

In addition to being five words or less, and must either be 95% probable, or be 10 times as likely as given the statement's reverse. Next, cross off duplicates. Each person reads their answers; if anyone got the same answer, they say "got it" and everyone who had that answer crosses it off. People may also say "challenge", if they think the answer isn't likely enough, the answer is a duplicate, or the answer is longer than 5 words. Challenges are resolved by the moderator. Answers that use synonyms of (as opposed to expanding definitions) or trivially different word choice for the same meaning, will count as duplicates at the moderator's discretion. If someone finds a template that produces unlimited answers, award two points for the template but don't count the individual answers. After all duplicates have been crossed off, score one point for each remaining answer.

Some other prompts: Jill is a banker. Paul drove there. There is a rock.

Comment author: handoflixue 05 April 2012 09:15:22PM 1 point [-]

I like this idea. I would say, though, that it teaches "how to make a specific statement", rather than "noticing WHEN you need to be more specific." If you're running in to groups that have trouble understanding what a specific statement is, or coming up with their own, this would be a very useful precursor exercise, though.

It's funny to realize that telling the entrepreneur to "be more specific" doesn't work, because you weren't specific about what "specific" means! :)

Comment author: Circaea 04 April 2012 04:39:09AM 4 points [-]

It seems like a common situation where this skill would be useful and is often lacking is where one person is an expert or semi-expert on a subject, but lacks conscious awareness of their criteria for expert decision-making. A concrete example of this that I have experienced is trying to explain to someone how I know confidently and in an instant something like "that tree is a kind of maple" or "that bird is a robin". Asking me this forces me to step back and start pointing out all the information I'm taking into account, which usually turns out to be possible, but surprisingly challenging.

The trouble with constructing an exercise based on the expertise of the student is that everyone's knowledge is idiosyncratic. And, with wide-speared knowledge, somebody usually has to pretend to be ignorant. I've seen an exercise used at job interviews, where they have the applicant "describe to a blind person how to tie shoelaces" (someone below had seen "teach me to sharpen a pencil", which has the same problems); I think that requires too much role-playing to be valid, although you could use an unfamiliar knot. A good exercise should never require anyone to fake ignorance. What I would like to see tested is an exercise where participants have to collaborate to win a game, rather than play-act at it.

So I would pick some totally arbitrarily domain of knowledge. I've personally spent a lot of time using field guides, so I have an opinion about that, but you could conceivably use anything with a ready-made compendium of information.

So, two participants are separated by a wall, such that they can hear each other perfectly, but not see each other.

Person A is presented with an organism (or several photos thereof) they are unlikely to have spent much time looking closely at (which, trust me, is usually just as good as something they have never seen before). Better still, if you have time, person A is shown a group of similar organisms. Personally, I'd just use weeds you find outside of the office, because they are free and no one ever looks at them closely. Person A also gets a ruler and magnifying glass (and, if it were up to me, several completely useless items to keep them from fixating on any one in particular).

Person B gets all the reference materials. (I'd be curious to see how it worked with and without a dichotomous key, and with photos vs. field marks, if those are options.) Together, they have to work out what Person A is looking at, or get as close as they are able, and say specifically why they arrived at their conclusion.

In order to win the game, you can't just say "it has a thingy" or "it's red" -- you have to be able to explain how you know what you know, and you are forced to arrive at a mutually intelligible vocabulary. I will caution that this will be an insanely difficult exercise for some people, and they might need some incentive to not give up too easily. I think it has to be hard in order to not seem excessively contrived.

The potential failure mode here is that any skills gained wouldn't generalize, even if the game really does require communicating concretely in order to win. I would definitely not expect people to be immediately better at pitching a start-up afterwards, but the hope is that they would come away with the experience of what it feels like to communicate concretely about a domain of information they have access to.

I'd love to test this on my own friends, actually, but I think they'd kill me.

Comment author: John_Maxwell_IV 04 April 2012 02:15:31AM *  4 points [-]

"Read through the comments, gather the LessWrong usernames of everyone who made a suggestion we tried or adopted, and email the list to Luke"

It's pretty easy to just let Luke do everything isn't it? (No snark meant; I noticed this tendency in myself when we were housemates and started actively trying to fight it.)


It seems possible that when people take personality tests, they just write down their perceptions of themselves. A much better way would be to think of specific examples (of times when they were on time or late for an appointment, for instance). Maybe if you got people to tally up specific examples, they would get a result the genuinely surprised them. Instructing people to search for examples in a way that didn't favor their preconceptions could improve results even more.

Reasons this idea could be good: personality tests are inherently interesting, and they are a fairly emotionally neutral discussion topic that doesn't risk excluding anyone. (If you started asking folks to brainstorm reasons why they were or were not a diligent employee, you might run into some problems.) You could even brand it as being an unusually accurate way to administrate a personality test.

Phil Goetz on personality tests:

[The main advantage of Myers-Briggs] is that it manages to classify people without insulting them. (This is accomplished by confounding dimensions: Instead of measuring one property of personality along one dimension, which leads to some scores being considered better than others, you subtract a measurement along one desirable property of personality from a measurement along another desirable property of personality, and call the result one dimension.)


This is a point against my idea if all the scientifically validated classification systems, like the Big Five, run the risk of insulting people.

Comment author: Stefie_K 04 April 2012 06:45:03AM 1 point [-]

This gets people thinking of specifics, but would it contrast being specific with failing to be specific, and make the students want to be specific in the future? I think that the students need that contrast just to appreciate what the issue is, and they need to see what they're doing as something that could apply to a broad set of situations in order to find occasions to behave this way in the future.

I suppose you could contrast your test with a personality test that doesn't use specifics, and that could supply the contrast. How would you supply the applicability?

I might be overestimating people's tendency to compartmentalize, but I doubt it. Once, when my parents were visiting my apartment before I got a microwave, my mother wanted to reheat some food in the ordinary oven. She asked what temperature I thought she should use; I didn't reheat food in the oven very often, but I suggested 300 degrees.

It took me several seconds more before it occurred to me: "Mom, you reheat food in the toaster oven at home all of the time. What temperature do you use then?"

Then she knew what temperature she should use. But she didn't automatically bridge the gap from "toaster oven" to "real oven," and I almost didn't, either. We're looking at a bigger gap, here, and a very strong tendency to be not-specific.

Comment author: RolfAndreassen 03 April 2012 06:08:29PM 4 points [-]

For moderately tech-savvy people who are not programmers: Write pseudocode to sort a list of numbers. Have a human, perhaps the moderator, execute the steps on a blackboard with a particular list. Repeat until you get the expected results. For extra credit, drill down on steps such as "exchange two numbers".

Ask for ways in which 2012 laws are better than 1912 laws. (Year is arbitrary.) Drill down on abstractions; for example, if "unions have more power" is given as an answer, ask for specific ways in which this is an improvement. For added weirdness, ask for examples of how the laws are worse now.

"Why did the United States secede from Britain?" Again, drill down on abstractions. If someone responds "Taxation without representation", ask what was being taxed. Allow Googling.

"Where do forks come from?" Trace all the inputs.

Comment author: thomblake 03 April 2012 06:01:37PM 4 points [-]

Exercise: Interview (or "be Paul Graham")

Example: (participants A and B)
A: I am a gardener.
B: What is a gardener?
A: It's my job.
B: No, what does a gardener do?
A: I maintain gardens.
B: What is an example of a task that you have to do?


2 participants act in roles of the Interviewer and Interviewee. The Interviewer asks questions to be answered honestly, ideally that would tend towards abstract answers like "What do you do for a living?" and "What did you study in school?". The Interviewer repeatedly asks for more specificity, until the responses are at the level of describing particular objects and tasks.

This goes on for a while, then the participants switch roles.

Rationale: Some of the game exercise ideas are great, but they don't seem to demonstrate using the skill in non-contrived situations. So this exercise is intended to overcome that problem by exposing non-(specificity/concreteness) in our actual communication.

I believe that this exercise can be performed between 2 people who have the barest grasp of the concept. While the skill of being specific needs to be learned, the skill of noticing that the other person isn't being specific should be much easier. After observing the game once, anyone should be able to ask some of the right sorts of questions to get the other person to be more specific. And then by repeated iteration, participants should be able to give more specific/concrete descriptions at the outset.


This does depend on the assumption that the skill "Ask for the right level of specificity" is easily / automatically learned.

A list of questions should be developed that are actually answerable for people in different walks of life (unemployed, did not attend school, etc.).

It might be too easy to cheat by being way too specific. But then, if you can do that, you might not need to learn the skill. Example: starting with "I push keys on the computer keyboard with the tips of my fingers, and characters appear on the screen corresponding to what I typed."


One person is the interviewee, several people are the interviewers. The interviewee needs to make themselves understood by each of the interviewers. This might help train new interviewers.

Comment author: TheOtherDave 03 April 2012 07:09:49PM 2 points [-]

When I did more job interviews, I was fond of the question "What does the job you'd most want here look like, in terms of what you'd actually be doing on a typical day?"

Comment author: thomblake 03 April 2012 06:25:20PM 2 points [-]

Inspired by people's failure to grasp at the obvious solutions to Harry's problem after Chapter 80, I posted an obvious exercise in case everyone's failed to notice it.

Comment author: [deleted] 03 April 2012 10:02:22AM 4 points [-]

An idea of an exercise, which actually involves some popular culture - which may make it more interesting for those involved. Agree on a set of books/movies/series/etc that can be used before starting, which participants are reasonably familiar with. Postulate a broad category of characters, e.g. "Good politicians". Brainstorm a list of characters who fit this category and put the list onto a piece of paper or a whiteboard. Take turns in trying to narrow down the list by making the category more specific and crossing some characters off until only one remains who fits the very specific description, e.g. (crudely speaking) "Good and careful politicians" eliminates Viserys but keeps Varys, Tyrion, Littlefinger etc. (A Song of Ice and Fire) Variations may include trying to eliminate only one character in each turn etc. This method allows to use complex and nuanced specifications, which situation is closer to IRL applications of "Be Specific", but still to maintain some bounds in terms of the limits of the story. The disagreements arising in whether a character fits the classification immediately create scope for further specification, until a statement is made specific enough for both parties to agree.

Comment author: scottw 03 April 2012 09:56:33AM *  4 points [-]

Exercise (preferably attempted by participants before the lesson):

Another variation on an old classic. Split the group into triads, and each group is given a bunch of words in a hat (a box may also be used, or a hollowed out rock of some sort). The words, preferably, are about half-way "up" the abstraction lattice; "red" is good, "Steve Irwin" and "Concepts" are not.

Each person in a triad rotates between three roles: guesser, hinter, or observer. The hinter has to get the guesser to say the word within a short time limit (30s? a minute?) without saying the word herself, without saying words that rhyme with the word, etc. The observer writes down all of the hints and guesses, in order, (the observer can be replaced by an audio recorder of some sort, if resources are available) and is instructed to try to figure out the pattern of what sorts of hints lead to the best guesses, and ultimately correct answers.

After each participant plays each role 3-6 times, have the groups meet and discuss what the best strategy was on the part of the hinter. Hopefully they will arrive at something like the lattice strategy without being hinted of it beforehand.

Note: There are bound to be plenty of tricksy ways of imparting the word without saying it but also without shooting up or down the lattice, like rhyming it. I cannot possibly list all of them, but playing this exercise out a few times should reveal enough of those clever work-arounds to ban them.

Comment author: Zaine 03 April 2012 05:47:34AM *  4 points [-]

If CMR mini-camp participants learned the skill through exposure, then perhaps an incentivized game executed in organically occurring scenarios that rewards those who recognize and do not practice non-specificity, would do the trick.

I'm thinking of a point-based game. It would occur either during a specified block of time, or on a specified day; everyone playing would begin with an equal number of points. During this specified period players would earn points pointing out abstract, non-specific utterances of other players' (including utterances operating on an illusion of transparency) by acting upon that utterance in a literal way; the one who said the utterance would then concede a point to the one who first pointed this out (or first acted upon the non-specific utterance), as long as it was fairly won. I'm thinking that if, "Head over to Anna" is met with someone offering their skull to Anna as a gift, that'd be going a bit far. The winner, the person with the most points at the end of the day, would receive some gift all would be motivated by, or an individualized gift of their choosing (e.g. multiple prize options). The game would be in effect during other activities.

Some examples:
-"I think we should all come together. [intended meaning: gather]" is met with someone walking into the non-specifist, or many overtly gathering in simultaneous motion around the non-specifist.
-"Perhaps everything's like that, or we could try maybe helping or fixing whatever might be causing it." is met by a mock stern, affronted accusation that the non-specifist is 'like that', or is egregiously guilty of whatever 'that' is, immediately followed by a mock abashed realization that the pointer-outer is, too, like that, and so the pointer-outer exclaims "Woe is me!" and mock faints.
-"Can you help, with the, uh, you know, um, the thing?" is met with someone grabbing a leaf on the ground or a chair or some such object that is obviously not what the non-specifist is referring to, lifting it up high, and innocently querying, "This?"

Edit: ITakeBets' post on orders of specificity brought to mind the possibility this game might be improved (once some familiarity is established) by requiring pointer-outer's to state why the statement was non-specific after they're done acting upon the statement's non-specificity. To use his example, stating, "I can't play basketball; I'm five foot six," would yield, "It was vague in that it lacked relevancy," after the pointer-outer had in a tone of mock shock said, "Goodness, me! You must have the condition that severely limits prestidigitation, rendering you unable to delicately flick your second through fifth metacarpals in unison against any opposing force! 'Tis a shame the condition only affects those of the adult height of sixty-six inches; if only you were an inch taller!"

I have some qualms about the game's potential negative effects on productivity, yet perhaps the probability of that can be mitigated by clever scheduling. The basic idea is participants won't want to lose points, but will want to gain points, so they will become more attentive to, and precise in, their speech and thoughts.

Comment author: Eliezer_Yudkowsky 03 April 2012 08:03:31AM 1 point [-]

Hm. I especially note the concept of handing out some sort of non-monetary gift at the end of the session to someone. I wonder if that would be productive or counterproductive...

Comment author: Zaine 03 April 2012 09:17:06AM *  1 point [-]

I wonder if that would be productive or counterproductive...

Are you wondering after the productivity of the gift idea in particular or the productivity of the concept as a whole? If it's the gift idea, then I don't see how it wouldn't be productive; a desirous prize would ensure all players apply themselves to the game, and dedicate themselves to learning the skill in the short term. If the game proceeds as expected, then all involved would have had a jolly good time; memories linked to a strong emotion are more memorable (learned this somewhere - no citation I can recall), so the positive fun emotion tied to the game, and thus the skill, will insure its memorability.

The idea as a whole, however, could indeed be quite counterproductive. Though I assume there would be some down time during the mini-camp? walking between sessions, meal times, pre-sleep socializing? The game might fit well during those periods, however it could just as well hinder socialization through added pressure and unnecessary competition.

Comment author: 2hp 16 April 2012 02:26:52AM 3 points [-]

This has obviously been a fun thought-exercise for many of you, but I think what's at heart here, the "skill" we're eager to develop via exercise, is just plain old good communication. In the entrepreneur example, the failure is not his inability to BE more specific, when asked. The failure is that he doesn't already KNOW he hasn't been specific enough. So, the skill we need is not how to muster specificity when Paul Graham asks for it. What we want is simply to be able to communicate abstract ideas clearly in the first place.

All these conscious exercises of ever-increasing specificity miss the point entirely.

I don't have a developed exercise idea, but I was inspired by the following, which I happened to have been reading earlier today.


That is the original proposal for the World Wide Web, written by Tim Berners Lee at CERN. To read that document in 2012 is, itself, an exercise both useful and relevant to these discussions. The document is chock full of descriptions of things that did not exist in 1989, but which have now become mundane and ubiquitous. Tim is in the role of the entrepreneur trying to explain what he wants to build and why. One can't help but be astounded by the ease with which Berners Lee renders clear enough description that we can recognize all the specific things that have arisen from his "sketch".

For example: "Meanwhile, several programs have been made exploring these ideas, both commercially and academically. Most of them use "hot spots" in documents, like icons, or highlighted phrases, as sensitive areas. touching a hot spot with a mouse brings up the relevant information, or expands the text on the screen to include it. Imagine, then, the references in this document, all being associated with the network address of the thing to which they referred, so that while reading this document you could skip to them with a click of the mouse."

This is exactly the kind of context where we want to help people to be as specific as possible, when they are trying to invent something.

Exercises in categorization or "lattice-travel" seem to miss the point entirely. What we want is simply for people to choose the best language for their audience. The only remedial program I can recommend for for that is trial and error.

Comment author: buybuydandavis 08 April 2012 10:38:17PM 3 points [-]

I'd suggest a game - I Don't Wanna.

The two sides are shown an end state considered to be the goal. One side writes directions to accomplish the goal, including whatever constraints he wants. His opponent tries to fail to accomplish the task while still following the directions an fulfilling the requirements.

The second player should always be able to win in absolute terms. His real goal is to be minimally obtuse - find the minimal distance between our usual priors and the priors he has to assume so that the directions don't complete the task.

For any of the rationality skills, probably the most important meta skill is to have some reliable way to evaluate how well you are doing. Just as EY talked about use cases, you need your test cases as well. As the years have gone by, I've noticed how the methodologies in engineering/software development are generally applicable to life.

My game is close to one you can see everyday in contracting. Company A contracts on a fixed price bid to fulfill the requirements spec of Company B. Company A wants to minimize it's cost for fulfilling the contract, so it prefers defensible interpretations of the requirements that minimize their cost. On the other side of that pancake, Company A tries to find a defensible interpretation of the contract that maximizes their value.

Comment author: epigeios 05 April 2012 08:24:55AM *  3 points [-]

Okay, so split into sets of 2 people (or, split into 2 teams, or even dynamic teams could work). Person A asks a simple personal question about person B (such as "do you have a girlfriend?" or "do you have a college degree?" or "do you prefer dogs or cats?"). Person B then tries to answer like the people in the video did, by telling an abstract related story, or by answering a different question contained within or related to the question (like "well, dolphins are really my favorite animal" or "college degrees aren't really an indication that someone is able to perform well in their field of expertise"). Person B tries to talk as much as possible in response to person A's question without actually answering person A's question. Person A then tries to redirect person B toward A's intended subject by asking different or more specific questions ("what about as a pet?"). In summary: Person B tries to avoid answering person A's question, and person A tries to force person B to be specific by asking the right questions.

And of course, things like the ladder of abstraction can (and should) be explained before starting the exercise so that the people have references to draw from to reach their goal (person A trying to get an answer, and person B trying to not give an answer while still answering)

The primary reason why skills can be transmissible from master to apprentice, but not replicable by exercises is because the skill in question has multiple difficulties associated with it which are disconnected at the level of the exercise. Most people are naturally (through karma/disposition or experience) capable of easily getting past a few of the difficulties, and have problems with a few of the others. These people have no idea that they were able to get past a few of the difficulties, because they weren't difficult.

The problem with exercises is that they tend to concentrate on certain difficulties. So, the people who lack the related experience, or have the karma where they find exceedingly difficult the difficulties not expressed in the exercise, are not going to be able to get anything immediately useful out of the exercise. And in addition, these people won't know why they weren't able to get anything out of the exercise, or even what could be done to help them.

You (Eliezer) addressed this subject in this post by providing examples of related concepts. The problem is that those related concepts are all true, and so are a few others you didn't mention, as well as a few others you aren't even aware of; and certain ones are true for certain people, and untrue for others.

Here are some difficulties you didn't mention:

  • Common people are emotionally driven. Sensationalist methods attract their attention, and fool them into thinking positive thoughts about the subject. These people who were presenting start-ups were not simply being non-specific, they were trying to pander to an audience (which they were taught to do in school). They weren't even trying to express their ideas when they first started talking.

  • Going deeper into concreteness, many of their ideas were still ideas: partially concrete in their minds, but not concrete for people who come from a different perspective. One skill here that is useful that isn't learning how to be concrete is instead learning how to interpret others' questions, to translate from one perspective to another.

  • When emotionally charged (as in a presentation), many people have the tendency to both respond to emotional triggers and try to trigger emotions in the audience. Through this, these people tend to go up the ladder of abstraction. Sometimes these people can benefit by just being aware that the audience (Paul Graham) is not trying to trigger them.

  • Many people are taught how to pander to an audience. Often, these people don't even know that they are pandering to an audience. Instead of teaching these people how to do the opposite, instead teach them both how to pander, and how to know when they are pandering to an audience, and follow up by teaching them how to know when to, and when not to, pander to an audience. For many people, after they learn this, the opposite side of learning how to be specific is easy.

  • All of the people in that Paul Graham office hours YouTube video seemed to me to be trying too hard to answer the specific questions asked. They interpreted the question to refer to one concept, and tried to convey that concept. These people would benefit from learning how to interpret questions from multiple angles. rather than trying to find the concept the questions is truly asking, they should be trying to find all of the concepts the questions is addressing. From there, they can pick the concept that they DO have a specific answer for, or determine that they don't have an answer, or don't understand the question.

Many exercises designed to help people get through difficulties like this would work better if they addressed the opposite. Get people to experience the problem; so that they can recognize it and try to find a solution, and become aware that it is indeed a problem. Don't get people to try to find a solution; because they usually aren't even aware of the problem, much less have a concrete understanding of it. For example,... oh, I just thought of a good exercise: [Idea moved to top so that it is the first thing people see. Not going to edit post to make it coherent in that order because I'm too lazy.]

Comment author: anemone42 04 April 2012 12:04:11PM *  3 points [-]

A simple exercise, borrowed from giraffe language / non-violent commenucation. Describe what happened in a way, even your worst enemy would have to agree with. This means sticking to what you saw, heard etc., without lumping things together or including judgements.

So, if your friend and you were to meet at the cafe at 13.00, and he showed up at 13.05, there's not much else you can say about that situation. You can't deduce a motive for his being late, and it wouldn't be wise to lump this being-late together with other being-lates.

Exercise based on this: 4 people work in 2 groups. A will tell B about some incident or period of her/his life. C will talk to D. Then B talks to A and D talks to C. Then the 2 groups merge, and B gets to retell A's words. Unless B is specific, A will have reason to correct summaries and judgements. Then A, C and D retell their partners words.

Comment author: fubarobfusco 04 April 2012 02:51:37PM *  6 points [-]

This means sticking to what you saw, heard etc., without lumping things together or including judgements.

So, if your friend and you were to meet at the cafe at 13.00, and he showed up at 13.05, there's not much else you can say about that situation. You can't deduce a motive for his being late, and it wouldn't be wise to lump this being-late together with other being-lates.

Go further. The friend showed up when the clock on the wall said 13:05; or my watch said 13:05; or whatever. After all, the friend's watch may be five minutes slow; or yours may be fast.

I suspect the bigger source of disagreement here, though, is whether 13:05 counts as "being late" for a social meeting booked for 13:00. This turns out to be extremely culturally dependent. Some cultures (and some people) value on-the-dot punctuality much more than others. To some, five minutes is a rounding error, and describing it as "being late" would be the equivalent of getting out your protractor to determine if you've been given a fair slice of pie: requiring that level of precision from someone indicates that you're either a crank, or looking for an excuse to have a disagreement with them.

So going from "we agreed to meet at 13:00; he showed up at 13:05" to "he is late" counts as a "judgement", too.

Comment author: Morendil 03 April 2012 08:53:16PM *  3 points [-]

A caveat - "specific" is not the same as "detailed". An exercise which teaches the latter may fail to hit the former.

"Detailed" is a listing of all your customers with a testimonial from each of them, many of which will be redundant, others irrelevant ("pretty home page design"). "Specific" picks out the particular thing that serves your customer better than the competition.

Being able to recognize that a given example is particularly illustrative - an examplar - is a subskill.

Comment author: Swimmy 03 April 2012 08:08:57PM 3 points [-]

I'm confused. Don't you already have your answer? Attendees picked it up by instructors asking attendees to be more specific about things they were discussing. Just turn that into an exercise: ask some questions about a subject that is suitably fuzzy in most people's heads (economics, philosophy, future prediction). Give feedback. Repeat until they get it.

And if you want something more specific:

Ask questions like, "Is a 'weak dollar' bad?" People will default to their cached thoughts. "Of course it's bad. It's weak, it should be strong." But nobody will actually say that out loud, because it's not very meaningful, so people will usually change to something vague that also sounds good: "It makes us more competitive." From there the specific path should be obvious. How, why? How are you judging "good"? Etc. Another good question from economics that trips people up: "The average child owes $X to the government at birth. Is this bad?" Or how about, "You find a clean harmonica lying on the ground with no way to find the owner. What is it worth to you?" A correct answer will not be in dollars, but in feelings. Students should consider what they can do with it, how those things would make them feel, and which option they would actually choose (which feelings they desire most).

Perhaps those questions are too technical, but I think they can be reasoned out from basic principles of how one thinks people will act under certain circumstances. The key point is that they trigger silly cached thoughts that people try to mask with vague language. They should at the very least learn to ask, "Good for whom?" and give some answers according to different groups of people.

(Eliezer, you should be able to generate more/better examples than I can, since you have apparently spent so much time telling people to be specific. But if the SIAI really wanted an outsider to design the whole lesson, I'm sure I or many other people could crank a lot more out.)

Unless you're asking for a way to replicate the skill without having a "master" who already has a sufficient idea of acceptable specific-ness, in which case I have no clue. Someone has to judge whether an answer is specific enough, and that someone has to have already mastered the skill.

Comment author: meeple 03 April 2012 02:47:51PM *  3 points [-]

Exercise: Add as many qualifiers as you can that do not make your statement irrelevant or false.

For example:

My startup is better than MixPanel

My startup is better than MixPanel at making revenue on day zero

My startup is better than MixPanel at making revenue on day zero when the economy is down

Well, never mind, that didn't work.

Comment author: Vaniver 03 April 2012 05:37:51AM 3 points [-]

Continuing with the "adapt a classic" suggestions:

Surely there's some way to adapt charades to this. You give someone an example of a complicated concept, and have them try to communicate that concept in as few examples as possible. We have similar problems with the twenty questions suggestion, though: a lot of specificity depends on deep knowledge of the subject matter. If you get a concept you or the guessers have never heard of, then you're dead in the water and that'll be frustrating.

The skill goals appear to be mostly "articulate the knowledge you have" and "model your interlocutor's knowledge." If I get a card that says "a startup that tracks viewing statistics and can combine them with parameters from the site to get detailed knowledge of how different classes of customers interact with the site", explaining that concept is going to be odd, and it may have been the communication failure between the Entrepreneur and Graham may have been that E didn't realize that G didn't know that they were about combining viewing statistics with user data. If the card only has one meaningful datapoint, things are too easy- the 'knowledge' and 'model' parts are done for you, and you just need to articulate- but if the card has lots of meaningful datapoints, things are too hard- now you need to figure out what the needle is in a haystack, probably in a field that's not relevant to you.

That suggests an alternative approach- give two people two different cards, with related but distinct concepts on them. Now, they have to figure out what the other person's concept is with as few examples as possible. Both players share whether an example fits their rule or not. If you score, consider including the number of examples the other person gives into the score, so that you have an incentive to ask questions with examples that both convey as much information about your concept as possible while also seeking to get as much information about their concept as possible.

For example, I might have the card that reads "A blue polygonal shape with less than five sides" while you might have the card that reads "a red polygonal shape with at least four sides."

I begin by saying "a blue pentagon fits my rule." You respond with "A blue pentagon does not fit my rule," and then follow up with "a red square fits my rule." I respond with "a red square does not fit my rule," then follow with "a red hexagon does not fit my rule." You respond with "A red hexagon does fit my rule," then follow with "A red triangle does not fit my rule." I respond that a red triangle does not fit my rule, and the game continues.

Once I think I have your rule, I write it down and stop providing examples. You can continue to provide as many examples as you like, until you also write down a rule. We then reveal rules, and then if scoring get points for guessing correctly, possibly with points taken away for every example provided.

This looks like it'll be obvious with rules based on geometric concepts, and similar mathematical objects so long as everyone is familiar with them. It should be extendable to fuzzier concepts as well (adding a "maybe fits my rule" or related answers will help).

Comment author: jschulter 04 April 2012 04:03:02AM 1 point [-]

This activity seems like it would tie in well with a unit on hypothesis and experiment generation as well- it reminds me of the 2-4-6 test. Perhaps have two different scoring rules: when trying to teach specificity, give points for getting your partner to guess; when teaching how to find the right hypotheses and tests, give points for guessing correctly.

Comment author: lukeprog 03 April 2012 08:31:29AM 11 points [-]

I, for one, am thrilled to see Eliezer writing another Sequence. :)

Comment author: fubarobfusco 03 April 2012 08:10:03AM *  6 points [-]

Exercise - refining descriptions.

Get three or four people sitting together. Place a large group of different items (30-50 small cheap plastic toys of the sort readily available in bulk from Oriental Trading Company, for instance) in the center of the group.

The exercise is to narrow down to a single toy by adding one detail at a time to a description. The description begins with the word "thingy," "toy," or a similarly vague word. Players take turns adding a single detail to the description, repeating it each time. (A detail is usually a single word, but a short phrase such as "with wheels" or "with spots" counts as one detail. However, "with yellow spots" would be two details — spots and yellow.) As an additional restriction, superlatives such as "biggest" or "greenest" are not allowed, because these implicitly compare each item to each other item.


A: Give me the thingy.
B. Give me the red thingy.
C: Give me the red thingy with wheels.
A: Give me the red and yellow thingy with wheels.
B: Give me the big red and yellow thingy with wheels.

Comment author: atorm 03 April 2012 01:47:33PM *  3 points [-]

A variation: Again, a large group of different items (I was thinking abstract images of colors and shapes, but toys is a good idea too) is visible to the group. One of the objects is selected and then each person writes a description that selects that object alone out of the set. The goal is to write the shortest description that can pick that particular object. Once everyone is done, answers are compared, and violations are sought: if one of the other objects fulfills all the requirements of someone's description, they are disqualified. Whoever has the shortest description that describes the chosen object and only the chosen object wins a point, and the game is repeated with another object.

Comment author: atorm 03 April 2012 01:52:01PM 1 point [-]

This adds a competitive/fun element (looking for violations in other descriptions), may widen ideas on how to describe something, and trains conciseness as well as specificity.

Comment author: dbaupp 04 April 2012 08:50:03AM 1 point [-]

(I like the idea, but I think the last few paragraphs could do with some editing: your meaning has been obscured :( )

Comment author: Daniel_Burfoot 05 April 2012 12:06:40AM *  5 points [-]

Notice that the entrepreneur's failure to convey his idea to PG was not, evidently, a failure of Thought - as you footnote, his system was actually pretty well-thought-out. Instead it was a failure of Communication. This distinction is important and shouldn't be blurred. Communication requires two parties, and a failure of Communication can't always be definitely attributed to one side or the other; maybe PG just wasn't asking the right questions. Or maybe he didn't care about understanding the idea at all but just wanted to probe the entrepreneur's ability to remain steady under pressure, which ability is obviously advantageous to would-be-CEOs.

Comment author: Vaniver 24 December 2012 08:02:04PM 2 points [-]

Zach Weinersmith describes Professor Liar. One player gets a field of expertise randomly drawn from two lists of 100 elements (the professor); the other players (the examiners) ask the professor questions, trying to expose that the professor is not in fact an expert on Feminist Mustache Theory. It seems like it could be readily adapted to focus on the specificity aspect of things, rather than the "convincingly bullshit" aspect of things.

Comment author: RomeoStevens 14 April 2012 10:43:30PM 2 points [-]

Have you tried playing Corrupt a Wish? No one has mentioned it by name yet.

Comment author: bliumchik 11 April 2012 09:55:08AM *  2 points [-]

Here’s a fairly simple one for thinking concretely about the abstraction lattice. Mine an encyclopedia article for topical words (i.e. omit “the” and “and” and their ilk – also omit duplicates, should be fairly easy to program for). Place each word on an index card and have the students arrange them on a large flat surface in order of abstraction – I’d have some large amusing goalposts at either end, but this is not strictly necessary. It should probably be acceptable for some words to be judged equally concrete.

Because this is a collaborative exercise, students will have to talk about what makes a word more or less abstract in order to justify their hypothesis that one word is more abstract than another word. If the concepts in this article had just been presented to them, particularly the section about superconcepts, I hope such discussions will help students become more adept at making those judgements. Using an encyclopedia article is only one of many ways to keep the words vaguely related to one another and hence more comparable – my instinct that this will help may not be accurate, in which case a random selection of X words from the dictionary will suffice. I expect that'd come out in testing.

(An alternate version splits the students into Team Abstract and Team Concrete, with the former responsible for seeing words that are more abstract than their current position and remedying this, and the latter with the inverse task. Then swap for a different lexicon. I’m not sure what effect that would have beyond or indeed specific to priming bias, and I’d be interested to see whether the alternate version consistently produces different results to the original version, and if so in what way, but that’s an entirely different rationality lesson, I think.)

Comment author: Douglas_Reay 09 April 2012 06:21:59PM *  2 points [-]

I once played a mage in a Live Action Role Playing adventure, and cast a spell to summon a water elemental, to fight on the side of the party against the monsters opposing us.

One of the GMs appeared, appropriately clothed, and proceeded to give the fastest lesson ever on being specific. Because, the way he played it, the water elemental would obey commands, but was malicious, and where possible would misinterpret the command, or do things I'd not yet specifically forbidden it to do.

elemental appears, moves to nearest party member and starts hitting her
Me: stop!
elemental stops walking, but carries on attacking with its arms
Me: stop attacking, don't do anything until I tell you to, then do only what I've told you to do
Me: those people over there points are the enemy - go attack them
elemental goes attacks a wall in the indicated area, then returns and stands on top of a party member, not doing anything but immobilising them

Comment author: Douglas_Reay 09 April 2012 06:09:29PM 2 points [-]

"Open with the concrete example, not the abstract explanation!"

Whether concrete-first or abstract-first works better may well be a cognitive style that varies quite a bit with culture.

Kamentz & Womser-Hacker, 2003, Defining Culture-Bound User Characteristics as a Starting-Point for the Design of Adaptive Learning Systems

Comment author: Vaniver 07 April 2012 10:02:59PM *  2 points [-]

During our discussion about Be Specific, majus had a good point:

Be Specific seems like a subskill of a more generic skill: fixing ontology problems. Going down a level helps you discover that you're talking about very different things when you use the word "connection"- if one person is thinking "interpersonal relationships," and the other person is thinking "communication mediated by an online application," they're not going to understand each other very well. Specific examples don't fix the ontological models, though, although they make them easier to fix by bringing the problem to your attention.

Another idea is trying to come up with a game where the players have different mappings from features to words. There might be a blue triangle that I say is a "gleam board" but you say is a "dull spike"- but my shininess words (gleam, dull, etc.) map to colors (blue, red, etc.) whereas your shininess words map to shapes (gleam -> square, dull -> triangle). With several different features and different words, the mappings can be nontrivial, and the challenge would be to figure out the mappings in as few examples as possible.

This works if you have examples that you can point to, but is weird if you try to do it just by words- if I say "gleam board" and think I mean a blue triangle and you hear "gleam board" and think I mean a red square, then unless we draw them out we can't figure out what's going on.

I also learned that explaining a variant of this game was trickier than I thought it would be; I should have just come up with an example to give people, rather than explaining how it worked. Oops. (We split into groups, and so I couldn't generate my group's rule, but I should have explained what kinds of rules were permissible with examples. Yes, I noticed the irony.)

That game appeared to work well (but way better with two people), but we didn't feel it fit Be Specific all that well. The particular variant I used was that the rules fit A_B_C, where the rules varied on what operator fit in the _. At least one of them had to be an equality or inequality, and the other could be an inequality, equality, or one of the four basic operations (+, -, *, /). (That means there are 65 possible rules.)

We also tried out this game, but with the students both guessing the questions and coming up with examples, which I predicted would work poorly and did work poorly. I think that with the teacher generating questions it works better- it seems useful at concept concretization, but not at the "pick concepts which communicate the concept boundaries well" skill.

Comment author: Psy-Kosh 04 April 2012 03:18:21PM 2 points [-]

Exercise idea that may simultaneously teach this and help illustrate notion of Locating the Hypothesis: A prize/treasure hunt in which you make them make you be specific. ("you", of course, being understood to mean whoever's teaching them.)

ie, hide some prize (money, for example) somewhere in the building or such. Tell them you will answer their questions about where the prize is, but they might have to work a bit to ask the right questions.

You: "Ask me where the prize is." Them: "Where is the prize?" You: "Somewhere. You're going to have to do better than that."

Them: "Is the prize in this room?" You: "yes"/"no" (depending on whether the prize actually is in the room, of course.)

Them: "which room is it in?" etc etc etc...

Comment author: Arran_Stirton 04 April 2012 05:27:41AM *  2 points [-]

Exercise/Game: Elevator-Pitch Descriptions

For two players. (P1 and P2)
Each player tries to describe a specific image for the other player to identify out of a larger set of images in 30 seconds.


  • Two different sets of images, and clones of these sets.
  • Something to stop P1 and P2 from seeing each other’s images. (e.g. a battle-ship style setup)
  • Clock/stop-watch that can take the time in seconds

Constructing the image sets:

The sets of images need to be of a moderate size (say 20 images) and the images themselves need to depict similar things. For example a set could consist of pictures of different monkeys, or of different (but similar) buildings.

(I read somewhere that it’s much harder for humans to tell the difference between two monkeys than two humans.)

The idea is for the images to be similar enough for any sufficiently abstract statement to include all of them, while also being different enough for a sufficiently specific statement to single out just one. Based on this it should be possible to fine tune the image sets to different levels of capability in being specific.

The images in each set should be numbered on the back (and its clones should be numbered in the same way) in order to allow quick identification of which image is which.

Setting up:

After being screened off from one another, each player is given a different set of images. Then they’re given an image from the clone of the other players set (so P1 will have a single image from P2’s set).

How to play:

P1 (after being given as much time as needed to think) tries and describe the image (s)he has from P2’s set in 30 seconds. After this P2 will attempt to work out which image P1 has from her/his set based on P1’s description. The roles are then reversed, P2 describes the image (s)he has from P1’s set and P1 tries to work out which image P2 has. Rinse and repeat the process until both players have identified the image the other player has.


Scoring would be done in a similar way to golf; the lowest score wins and every time a player isn’t successful in getting the other player to pick out the right image they get a point.


Not allowing the player to see the images they’re trying to differentiate the image they’re describing from forces them to say what makes their particular image different from all other images. What it is that makes their monkey different from all other monkeys, rather than just the rest of the monkeys in the set. Along those lines players could occasionally get an image that isn’t part of the other players set, meaning that they’ll have to cover the basics of “It’s a monkey” and so on just to be sure.

Hopefully this will help improve conscientiousness too. By giving the players as much time as they need to think before they give their 30 second description it should get them into the habit of thinking carefully about how they’re going to say something before they actually say it.

(I’ve noticed part of the reason people fail to be specific is because they’re trying to rush their description of things.)

Phew... that took way longer than I thought it would to write.

Comment author: ESRogs 03 April 2012 10:34:56PM 2 points [-]

I did not understand the distinction being made between a ladder and a lattice. Is the idea that the lattice is multi-dimensional? If so why was Eliezer still talking about the top and the bottom rather than some point and the origin?

Comment author: fiddlemath 03 April 2012 11:56:23PM *  4 points [-]

He's referring to the mathematical formalism.

Sets of elements, ordered by the subset relationship, are a good example of a lattice. Lattices have a specific "top" and "bottom". Among the subsets of the letters in the alphabet, the empty set is "bottom", and the entire alphabet is "top".

The particular point of calling it a "lattice" rather than a ladder is that there are many ways to be more specific; of these, not all are more or less specific than each other. (Formally: you can have a < c and b < c such that neither a < b, a = b, nor a > b.)

Comment author: Sniffnoy 04 April 2012 12:22:46AM 1 point [-]

Syntax note: Link is broken due to not escaping the closing parenthesis in the URL.

Comment author: Stefie_K 03 April 2012 10:00:31PM 2 points [-]

Hi! I've been lurking for a bit.

It looks to me like one thing that would help would be to get the people you're teaching to get frustrated enough to spontaneously say "Be specific!" on their own. If you can get them to associate a feeling of frustration with certain situations, the emotional reaction could reinforce the cognitive skills they're developing.

Specific scenarios can be based off of actual conversations like the one Eliezer presented in his post. Here's an example, based on Eliezer's example:

Sample Exercise:

The student must decide whether to invest in Company X or MixPanel. They will be investing $1 million of their company's money in one (and only one) of them; their only choice is which. They must be prepared to justify this decision to their boss, or they will be fired (and possibly have to deal with a lawsuit? Try to ramp up consequences).

Have a "boss" be there to help steer the conversation if the student is too happy with something unspecific. For the most part, though, the boss will be silent, and perhaps look unhappy.

The "entrepreneur" (now a teacher, not a student) speaks as though s/he really wants the student to invest in Company X. However, s/he is never specific. The exact language is taken/adapted from the recording of the original exercise.

In order to make sure that this exercise isn't derailed by focusing on money than by the actual difference between the companies' products, the boss can point out that even if the funding and/or number of customers is good now, that is not guaranteed to continue, and that they need information on what exactly makes the two companies' products different, and Company X's product better.

I would provide some scripted language, but there isn't enough continuous dialogue in the post for me to do much more than copy and paste it here.

Exercises like this could be designed from scratch, but if there are enough recordings of actual situations like the one Eliezer used, that could be more efficient, and quite possibly more realistic. Ironically but usefully, this approach to designing specific exercises of this type is made easier by the apparent disconnect between what the student-entrepreneur said and what Paul and Harj were asking. If there's usually a disconnect, then that makes it easier for a script to fit a conversation being directed by a student-investor, and it won't sound much less natural than the original situation.

One issue with this is that you would need to make sure that the scenario in the exercise doesn't match up too closely with the student's own background. In the example that Eliezer posted, for instance, an example involving software solutions would be a bad choice for the entrepreneur-student, because he'd be too used to hearing unspecific explanations and solutions of that particular type, and might well not recognize them as a problem. This requirement would mean that a lot of scenarios would need to be written (in order to ensure enough variety), and that people would need to be matched up with the particular scenario they'd be put in.

What do you think? I could also try to come up with from-scratch scenarios, but my ideas had a tendency to be vague in a way that's different from how people would realistically be vague. This exercise is much more useful if people seem sincerely not-specific, rather than trying to be not-specific, both because it avoids the impression that this is just a game (and therefore possibly not something to be frustrated by), and because realistic scenarios make it easier to recognize these situations in real life.

Comment author: John_Maxwell_IV 04 April 2012 04:31:57AM 2 points [-]

The "entrepreneur" (now a teacher, not a student) speaks as though s/he really wants the student to invest in Company X. However, s/he is never specific. The exact language is taken/adapted from the recording of the original exercise.

I'd be a little leery of embarrassing those guys even more than they've already been embarrassed.

Comment author: selylindi 03 April 2012 09:29:36PM *  2 points [-]

(These are not relevant to the skill of the week. I misinterpreted the instructions. I'll leave this comment up, though, because I enjoyed these games/exercises and others might as well.)

1) Play Wits and Wagers. This is an unoriginal suggestion and probably not eligible for a reward, but nevertheless it's a fun, socially rewarding, accessible, large collection of Fermi questions about which Bayesian updates based on vague real-world knowledge can be readily applied.

Variation 1 Add the additional step that every player must explain at least one reason that they think influences the odds on estimates of the answer.

Variation 2 To practice resistance to anchoring (assuming that's even possible), in each round one player is first told the question privately, then makes a public estimate. Then the other players hear the question and answer normally. After all estimates are revealed, the first player is allowed to "back out" and retrieve his or her ante.

2) In the Austin meetup, and later as a private game between my SO and I, we played my own variant of the 2-4-6 puzzle. I broadened the scope to "a sequence of any three words" in fairness to people who just don't like math, and renamed the game Trilogy. One player picks a rule simple enough that it can be applied fairly and consistently; this can be subjective, of course, so remind players that the goal is to make the game fun and somewhat challenging, not fiendish. That player then gives an example sequence of three words that pass (or fail) the test. Then the guessing player guesses sequences of three words and the rulemaker tells whether the sequences pass or fail the rule. It can be opened up to competitive group play by letting each player propose a test sequence, one at a time, in turn. The player may guess the rule instead of proposing a test sequence, and if correct he or she wins, and if incorrect he or she loses.

Comment author: vinayak 03 April 2012 07:36:12PM *  2 points [-]

How about this:

People are divided into pairs. Say A and B are in one pair. A gets a map of something that's fairly complex but not too complex. For example, an apartment with a sufficiently large number of rooms. A's task is to describe this to B. Once A and B are both satisfied with the description, B is asked questions about the place the map represented. Here are examples of questions that could be asked:

How many left-turns do you need to make to go from the master bed room to the kitchen?

Which one is the washroom nearest to the game room?

You are sitting in room1 and you want to go to room2. You have some guests sitting in room3 and you want to avoid them. Can you still manage to reach room2?

You can also just simulate the story about Y Combinator and Paul Graham. Show a new web-service to person A and ask him to describe it to person B. Finally ask B questions about the web service.

In both cases, the accuracy with which B answers the questions is directly proportional to the quality of A's description.

I think two variants can be tried. In the first one, A does not know what questions will be given to B. In the second one, he does, but he is prohibited from directly including the answers as a part of his description.

Comment author: Morendil 03 April 2012 07:19:09AM *  2 points [-]

Exercise - characterization. Form triads: one is the moderator, one the target, one the observer. Moderator asks observer "name one thing that struck you about <target>'s character". Observer goes, for instance: "she's funny". Moderator asks for specifics: "what particular things have you observed about <target> to make you say that?". Alternately, ask moderator to pick up on particular observations ("she has this way of pushing her glasses up her nose") and move in the other direction ("what does that tell you about her, if anything"). After a few minutes, switch roles.

This could be a good icebreaker, or a useful adjunct in a module on social skills.

Comment author: Vaniver 03 April 2012 05:13:54AM 2 points [-]

It seems to me like there's a twenty-questions style rule-based game waiting to be developed here. That is, rather than asking twenty questions about an object ("is it an animal?") you ask them about a concept.

There are a few challenges- the first is that natural divisions about concepts are unfamiliar, they're probably fuzzy instead of binary, and you need fairly deep conceptual knowledge of the thing in question to find it. For example, if the concept is "Athenian direct democracy," and the first question is "does it relate to history?" the answer is yes but that could easily lead down the wrong trail.

So maybe use a guided search progress. Show the students pre-generated questions and answers, and have them come up with both a general rule that will fit the category and an example of that category, for both the section just ruled in and ruled out. For example, we start off with a blank slate, and see the question "Does it relate to history?" with the answer "Yes." I say that the "in" category is close to the rule "any concept associated with history", an example is "the movement of troops during the American Civil War," and the "out" category is "any concept not associated with history", an example is "the 3 by 3 identity matrix."

A second question is asked- "Does it relate to economics?" and the answer is "No." My "in" category is now "non-economic concepts associated with history," an example is "universal suffrage," and my "out" category is "economic history," and an example is "the exchange rates of private currencies in the United States in 1850."

Having both in and out categories seems like it helps develop imagination in multiple directions at once, and have the descriptions building on each other (with the attendant change in concepts and examples) seems like it will simulate an organic conversation better.

I don't know how reliant this method is on a good question-answer trajectory. It seems like it'll make a big difference, but it also seems like decent trajectories should be fairly easy to generate.

Comment author: Vaniver 03 April 2012 04:58:05AM 2 points [-]

This is related to the incredibly important skill, search for the historical causes of your thoughts, rather than their justifications.

Isn't this not recommended by CBT? Everything I've read has been present-focused or forward-focused, whereas Freudian therapy is typically past-focused ("ok, we've figured out what you should do next time" vs. "ok, we've figured out who you should blame").

Comment author: ciphergoth 03 April 2012 06:51:45AM 6 points [-]

CBT focusses on the immediate causes, not the long-standing causes. "I'm feeling anxious because I've just got an email from my boss and it makes me worry that he's angry", not "I'm feeling anxious because of my troubled relationship with my nursery carers".

Comment author: Vaniver 04 April 2012 07:43:13PM 2 points [-]

Right. The full example he gave is:

Contrast also "It bothers me when you leave soda cans on the table" vs. "You're such a slob, stop being such a slob." Or contrast: "I'm upset" -> "I'm upset because I think the other person is looking down on me" -> "I'm upset because the person's tone of voice sounds like people who looked down on me in high school". This is related to the incredibly important skill, search for the historical causes of your thoughts, rather than their justifications.

The arrows implied a progression to me, and the mention of "historical causes" in that context seems like "original causes" rather than "examples of this occurring in the past."

Comment author: NancyLebovitz 03 April 2012 06:46:01AM 3 points [-]

There's at least a recent past aspect to CBT. If you say "Everyone at work hates me", you'll be asked for specific evidence that specific people (and how many of them compared to all the people at your job) hate you.

Comment author: Louie 03 April 2012 06:57:17AM *  2 points [-]

I think the bigger difference between CBT and psychoanalysis is something like, CBT: "Your feelings are the residue of your thoughts, many of which are totally wrong and should be countered by your therapist and you because human brains are horribly biased." vs, Psychoanalysis: "Your feelings are a true reflection of what an awful, corrupt, contemptible, morally bankrupt human being you are. As your therapist, I will agree with and validate anything you believe about yourself since anything you report about yourself must be true by definition."

CBT still works with specific past instances of your emotions to chart feelings into thoughts. It's good to do that so you can see clearly that thoughts always proceeded your feelings about a matter.... and also to see what the content of the thoughts are if they are, sneaky, "automatic" thoughts.

For example, "Jill made me sad." might be examined and reframed as "My automatic thought that hearing I was wrong about what day the garbage was picked up made me think: I'm wrong, therefore, I'm stupid, therefore, I'm worthless, therefore I'm sad. Those were all my highly-optimized and compressed thoughts which executed so fast... in such well-worn pathways... that I didn't even notice them. So my thoughts about that made me feel sad, not Jill."

Comment author: Eliezer_Yudkowsky 03 April 2012 02:37:45AM 2 points [-]

Further notes on the Prize:

"Figure out a way to X" is advice, not a prize-eligible suggestion - if your comment is "Figure out a way to X..." and someone else replies with a suggested way to actually do X, they get a prize and you don't. (This may sound harsh, but we already have lots of goals - we don't need help coming up with goals - we need exercises that actually achieve those goals.)

Please don't overlook the value of including at least one sample problem or sample use-case! If it's clear how to implement your suggestion on our end, it's prize-eligible even without any sample cases - but we may not understand what you mean, and working out a single sample case is likely to help you think clearly about the problem. (See also: "Be Specific" and Illusion of Transparency.)

Or to take the example of sunk costs:

"Figure out a way to get people to think consequentialistically about sunk costs" is not prize-eligible.

"Have them transform sunk costs into purchased options" would be prize-eligible if we had any idea what you meant by that.

What we're looking for is more along the lines of:

"Give them sample scenarios like *Paul is 3 years into completing his 4-year PhD in Obscure Egyptian Poetry, and is wondering if the improved salary prospects are worth it' and have them list the exercise cost of the option ('One more year of work') on one line, and the purchased benefit in terms of future events ('3% chance of getting a low-paying teaching position') on another line."

This would definitely be eligible for the prize... but our feedback with respect to "Having people writing things on paper by themselves" is sufficiently discouraging that we probably wouldn't find the idea worth testing if there were any open suggestions for 2-person activities, or anything other than "write things on a piece of paper by yourself".

If someone at CMR took that essential idea of describing a sunk-cost scenario in terms of purchased options, and transformed it into a two-person activity, we would credit the original suggester with a successful suggestion.

Nonetheless, turning all these exercises into non-boring activities is the part of the problem we most need help with. If someone else transformed it into a two-person activity in a reply to your comment, and we kept their final form, we'd split the prize between the two of you. If any more complex discussions go on, we reserve the right to do percentage allocations of credit on an arbitrary basis.

Comment author: ciphergoth 03 April 2012 06:49:32AM 8 points [-]

Please don't overlook the value of including at least one sample problem or sample use-case!

The irony of having to spell that out on this post is killing me.

Comment author: Viliam_Bur 03 April 2012 02:15:58PM 6 points [-]

The irony of having to spell that out on this post is killing me.

How specifically is it killing you? :D

Comment author: MixedNuts 09 April 2012 04:42:37PM 3 points [-]

Softly, with his song.

Comment author: NancyLebovitz 03 April 2012 08:41:50AM 2 points [-]

That's what I was thinking, more or less. A typology of the more commonly needed sorts of specificity would probably be very useful.

Comment author: Armok_GoB 04 April 2012 05:54:58PM 3 points [-]

The first thing that comes to mind is "write erotic fiction", but that has obvious social problems.

Comment author: TheOtherDave 04 April 2012 06:59:02PM 5 points [-]

When it comes to specificity erotic non-fiction is even better, but the social problems are greater.

Comment author: handoflixue 05 April 2012 09:06:32PM 1 point [-]

Why specifically "erotic", and then why "fiction"? What's wrong with "write about what you had for lunch" or "write about your commute to get here"? I think anyone who can write a page on those has a decent sense of how to be specific (although this does NOT train the 5-second-skill of noticing WHEN you need to be specific)

Comment author: Nornagest 05 April 2012 09:12:10PM 4 points [-]

I don't think it'd necessarily need to be erotic as such, but the "show, don't tell" concept (a subset of "be specific") is, if not quite specific to fiction, at least a lot easier to screw up there. If you're writing about your lunch or your commute to work, you've already got a sense of the salient details and probably won't be tempted to hide behind coarse descriptions of your emotional state; neither's necessarily true if you're writing about some fictional character.

Comment author: Swimmer963 14 April 2012 02:28:12AM 1 point [-]

I would say 'fiction' because you're forced to come up with details yourself, and 'erotic' because that's where most people, including lots of authors, shy away from specific details...the memory of painful making yourself write very explicitly would be a more memorable lesson than writing about something innocuous like, I don't know, two siblings playing Lego.

Comment author: JackV 04 April 2012 01:42:19PM 2 points [-]

I've been reading the answers and trying to put words into what I want to say. Ideally people will experience not just being more specific, but experience that when they're more specific, they immedaitely communicate more effectively.

For instance, think of three or four topics people probably have an opinion on, starting with innocuous (do you like movie X) and going on to controvertial (what do you think of abortion). Either have a list in advance, or ask people for examples. Perhaps have a shortlist and let people choose, or suggest something else if they really want?

I picked the movie example because it's something people usually feel happy to talk about, but can be very invested in their opinion of. Ideally it's something people will immediately disagree about. I don't think this is difficult -- in a group of 10, I'd expect to name only one or two movies before people disagreed, even though social pressure usually means they won't immediately say so.

Step 1 Establish that people disagree, and find it hard to come to an agreement. This should take about 30s. People will hopefully "agree to disagree" but not actually understand each other's position. Eg. "Starwars was great, it was so exciting." "Starwars was boring and sucked and didn't make any sense."

Step 2 Ask WHAT people like about it. Encourage people to give specific examples at first ("eg. I loved it when Luke did X") and then draw generalisations from it ("I really empathised with Luke and I was excited that he won" "I've read stories about farmboys who became heroes before, I already know what happens, bring me some intellecutal psychological fare instead"). Emphasise that everyone is on the same side, and they shouldn't worry about being embarrassed or being "wrong".

Step 3 Establish that (probably) they interpreted what the other person said in terms of what they were thinking (eg. "How can blowing up a spaceship be boring") when actually the other person was thinking about something they hadn't thought of (eg. "OK, I guess if you care about the physics, it would be annoying that they are completely and utterly made up, it just never occurred to me that anyone would worry about that.")

I may be hoping too much, but this is definitely the sort of process I've gone through to rapidly reach an understanding with someone when we previously differed a lot, and for some simple examples, it doesn't seem too much to hope we can do so that rapidly. Now, go through the process with two-four statements, ending with something fairly controvertial.

Hopefully (this is pure speculation, I've not tried it), giving specific examples will lead to people actually reaching understandings, imprinting the experience as a positive and successful one. Then encourage people to say "Can you give me an example of when [bad thing] would be as bad as you feel" as often as possible. Give examples where being specific is more persuasive (eg. "We value quality" vs "We aim for as few bugs as possible" vs "We triage bug reports as they come in. All bugs we decide to fix are fixed before the next version is released" or "we will close loopholes in the tax code" vs "we will remove the tax exempion on X"), and encourage people to shout out more.

Comment author: John_Maxwell_IV 03 April 2012 06:11:21PM *  2 points [-]

But you can see the basis for the hope that - after a fair amount more work - we'll be able to offer a 2-day course for YCombinator entrepreneurs that eliminates 50% of the overhead from their conversations with Paul Graham.

That sounds like an awesome idea!

They aren't the only incubator in the Valley either.

Can I submit this to Hacker News with an inflammatory headline based on this quote, or is it too soon?

Comment author: JoeW 03 April 2012 06:49:09AM 2 points [-]

The Center for Modern Rationality is now offering prizes for suggested exercises:

$50 for each exercise promising enough that we test it during a Saturday session. A $500 prize for any exercise which actually seems to work (as in, we decide to adopt it into the unit after testing).

Hmm. I'd understood that there was some pretty convincing evidence that offering cash incentives like this is counterproductive - it decreases creativity and effectiveness of troubleshooting. I don't have scholarly cites handy (I know there was a CMU/LSE study) but this is popularised in Dan Pink's "Drive". (There's a TED talk and a great RSAnimate video on this.)

Comment author: undermind 03 April 2012 02:32:16PM 5 points [-]

I was interpreting this not as "do this because we will pay you", but rather as a way of signalling how much CMR cares about this being done well and quickly.

Comment author: Mark_Eichenlaub 03 April 2012 10:54:45AM *  4 points [-]

In the RSAnimate talk, Pink cited research that found that once people were doing the task (presumably under time pressure), higher incentives produced worse results.

That's different from using an incentive to get people to come do the task at all.

Comment author: John_Maxwell_IV 04 April 2012 04:29:12AM 2 points [-]

I noticed a similar flaw in some studies Dan Areily was citing to prove that incentive pay didn't work, and e-mailed him about it. He didn't have any counterarguments when he responded to me, but he did keep citing the studies IIRC...

Comment author: Eliezer_Yudkowsky 03 April 2012 08:01:22AM 2 points [-]

Not offering money wasn't working, and if there are a few creative money-positive people reading this who try to pick up the free $500 bills, hopefully that's enough.

Comment author: Illano 04 April 2012 02:54:02PM 1 point [-]

Assuming that you want to keep the exercise somewhat entertaining, modifying a game like balderdash to start with dilbert-esque executive speak and move towards ever more specific levels could provide a fun, easy to understand method to practice moving up and down in levels of specificity.

As to how this would work in practice, have everyone present get into groups of 5-10 (or however many are sitting at a table) and give everyone a key phrase of execu-speak, such as "Energize the end-user experience." Then, everyone writes down a short, but more specific description of what the phrase really means. All of the player-generated statements are read aloud to the table (which will also display how vague the initial phrase was, since everyone should have vastly different answers), and the table votes on which is the best written response, and authors get a point for each vote they receive. Then, the exercise is repeated with the phrase the group selected as the best, for several rounds, or until everyone's responses are approximately the same, showing that a undisputable level of specificity has been reached.

For fun, you could run the exercise in reverse as well, giving a very concrete example and challenging people to go the opposite way, and give a less specific example phrase.

Comment author: handoflixue 05 April 2012 09:32:51PM 2 points [-]

Then, the exercise is repeated [...] until everyone's responses are approximately the same

In the real world, if the client wants X, it does not matter that ALL the engineers agree that the statement really says Y. Doubly so since engineers and clients often seem to speak two different languages.

It's far more useful to learn what the CLIENT wants, not what all the engineers THINK she wants. In fact, learning how to come to this sort of false conclusion seems exactly opposite of what Eliezer wants to teach - you now have a group of engineers convinced that they know what this overly-general statement means, despite never having checked back in with the client!

... of course, for a twist, do this, and then tell them that they're WRONG and the client actually wanted X. Punish them for not thinking to go and actually consult the client. It produces frustration and ill-will (the presenter has just tricked them), but with a receptive audience you do potentially teach a very potent lesson.

Comment author: wirov 05 April 2012 09:21:10AM *  1 point [-]

What I like about this exercise is, that it doesn't require any teacher/judge/… who decides what is specific enough (and may thus cause frustration in participants who disagree). Instead, the exercise ends when everyone agrees on the statements meaning – i.e. when the statement is "specific enough".

Comment author: meeple 03 April 2012 03:29:00PM *  1 point [-]

Imagine someone really really hyper and shortsighted is answering the question:

Why is your product better than mix-panel?


... or tone it down a bit...

Because I can like, put totally awesome stuff like BUNNIES on my version of the app!

... and then maybe take that statement and generalize it a little bit...

Because users can customize their version of the app.

Comment author: TheOtherDave 03 April 2012 04:04:46PM 3 points [-]

With my professional hat on, I would look askance at that last version and ask "Why is that feature a benefit? What users are we talking about, and what customizations will they be able to do, and how do those customizations make the app better for those users?"

Comment author: freyley 26 April 2012 05:20:47PM 1 point [-]

Within the domain of building-a-system, paper prototyping/wireframing teaches people to be specific with their ideas. It's only helpful when your ideas are "I want there to be this kind of thing" and then putting it on paper creates the specifics in your head.

Comment author: Vulture 24 April 2012 09:59:27PM 1 point [-]

I thought of a game for this called "Functionality Telephone"

So you divide people into pairs, and in each pair one person is the Manager, and the other person is the Designer. The Manager recieves a card with some sort of functionality pritned on it, like "Can hold a gram of water without leaking", or "Can have pebbles thrown at it and remain standing", or some other easily testable function. It will also have some taboo words, like (for the water example above) water, waterproof, spill, leak, etc. The Designer will have a bunch of legos or some such.

The game, as you may have guessed, is that the Manager has to give the Designer verbal instructions, without using any of the taboo words, to build something with the legos which will fulfill the functionality. Then the Manager will leave the room, and the Designer has to try to build something which follows the manager's instructions. When the Designers are done, the Managers return to the room and reveal their functionality cards, and then they and the Designer get to watch as their "product" is tested. Fun times are had by all.

The idea here is that the Manager is forced to describe their functionality further down the abstraction ladder than the form they recieve it in, in a lucid and detailed way.

Comment author: JohnWittle 09 April 2012 04:41:25PM 1 point [-]

Nitpick: "You can say more truths about apple2, like "apple2 is dark red", then you can say that is true of all apples."

The thirteenth word of this sentence should be 'than' rather than 'then'.

Comment author: NancyLebovitz 07 April 2012 08:54:27AM 1 point [-]

Given our present teaching technology, this skill seems transmissible from master to apprentice, but not yet replicable by exercises.

Transmission may be a lot of how specificity has to be taught, since it's about comparing what's in your mind that you're trying to communicate to what's actually being received by someone else's mind, and the lesson is clearer if it's something you actually want to communicate-- that is, not an exercise.

Comment author: Benquo 08 April 2012 04:08:57PM 1 point [-]

Computer programming has lots of replicable tasks, and with any kind of meaningful feedback it's quite good at "be precise," which is the abstract version of "be specific."

Comment author: NancyLebovitz 08 April 2012 04:44:36PM 0 points [-]

That doesn't seem like the kind of precision most people have trouble with, though teaching it explicitly might help people who have trouble getting started on programming.

Do you think knowing how to program helps people be sufficiently explicit about other subjects?

Comment author: Benquo 08 April 2012 08:49:53PM 0 points [-]

Probably not, because of compartmentalization.

I wonder, though, if there is some sort of similar activity that:

  • Doesn't require expert instructors (some feedback is obvious or automatic)
  • Deals with less mathematical and more "real world" problems
  • Builds skills that transfer well to other domains.
Comment author: handoflixue 09 April 2012 07:51:37PM 0 points [-]

I wonder, though, if there is some sort of similar activity

Management, which is at times remarkably similar to the "malicious idiot" game...

Comment author: imajunryou 04 April 2012 11:37:05PM 1 point [-]

Action exercise:

Instructor provides a set of somewhat vague tasks to a few groups or individuals ("Draw me a tire swing" / "Write me a poem about fish" / "Tell me about [yourself | your selves]"). The instructor has a specific interpretation in mind, in advance ("An old tire from a dump truck, tied via blue-painted rope to hang horizontal to the ground, 3 feet from the ground, from a large branch of a willow tree" / "A haiku about salmon and how they are unpleasant when used as projectiles" / "Your favorite hobby, color, flavor of ice cream, but not physical characteristics or past job history"). These tasks are mixed among more specific tasks, whose initial description more precisely matches their intention ("Draw me a series of five overlapping lines that connect end-to-end, and form a five-pointed star").

The groups are set loose to perform their tasks, and are able to ask for further details. The results are scored on proximity to the intended result and perhaps on speed.

Performed in rounds, it would help to pick out when one is speaking too abstractly about something that, on the surface, sounds simple enough (How much detail do you really need to draw a tire swing?!).

Possible Trigger: After a few rounds to get a feel for how it should work, break groups up into sub-groups, where they take turns providing and performing these minor tasks. Have those providing write down their intention up front, and the performer record any questions they ask for clarification, so that it can be compared with results. The intention, clarifying questions, and results can be used to reinforce the desire to be specific up front by scoring based on proximity of result to intention and also based on the number of clarifying questions needed to create the result (with the ideal being having an intention specific enough that fewer clarifying questions are needed to create a result with close proximity).

A narrower set of tasks that would require less setup: Have two sets of colored sticks (straws from a dollar store would suffice) that match each other in color, length and quantity. Group A arranges one set in a particular arrangement, out of sight of group B. Group A provides an initial description of the arrangement in as much detail as they feel is necessary. Group B attempts to recreate the arrangement with their own set. Group A gets a score for the similarity of the result of B to their own arrangement.

Group B is then able to ask clarifying questions, and adjust their arrangement as they seem fit. Upon satisfaction, they are scored based on their result's similarity to group B's initial arrangement. Only after group B has been scored should they be able to see group A's initial arrangement. It would probably be handy to only allow a certain number of clarifying questions, perhaps X questions per round for N rounds of questioning with group B modifying their arrangement as needed between rounds.

Comment author: handoflixue 05 April 2012 08:56:01PM 2 points [-]

I like the idea of "provide abstract instructions, and leave it up to the students to realize that this is too general, and that they need to ask for more information."

I would say, however, that usually when you need more information, you're aware of that need. In Eliezer's example, Paul can automatically recognize that he needs more information, because he doesn't have enough information for his own task.

The very simple example would be to split in to pairs: a describer and a picker. The describer is given a picture of the piece they need, and a quick description to read (print these ahead of time!): i.e. they might need a blue 2x8x1 block, but they're told to ask for "a long block". The picker then has to realize they don't have sufficient information, and ask to clarify.

It's teaching a VERY basic level of "realize I need more information, and then ask for it" - after the first example, every student will REALIZE what they need to be asking (size + color), so there's minimal sense of discovery/novelty. I think you would only be able to spend 5-10 minutes on it before adults got bored, but you could probably go a bit longer dealing with young kids.

I chose Legos specifically because they don't have a ton of variables, and it's very quick to hold up a piece and get told "no", which I think is a big help here. You don't want someone to spend 5 minutes drawing a picture only to discover they got it wrong, because that creates a big negative, a lot of frustration that you don't want in a quick-and-basic exercise like this. Save frustration for the bigger challenges, when you want to illustrate the cost of failing at this skill :)

This exercise would mostly serve to prime the audience's thinking in the right direction, and as a very mild "success spiral". You would definitely need to lead-in to a more complex and challenging example, to keep the audience engaged

Comment author: Benquo 04 April 2012 07:40:49PM 1 point [-]

Here's a game that might be a little more vivid, and easier to set up, than some of the games that are strictly verbal games. Call it "Tech Support."

There are two players, separated by a screen or barrier so they can't see each other, point, or gesture, only communicate by words.

One player is the Customer. They are trying to figure out how to use or perform some operation on a physical object. (Preferably one they actually don't know how to use!)

The other player is the Expert. They are trying to help the Customer use the object.

A simple musical instrument like a recorder would be a great thing to start with.

"I see two pieces."

"Oh, then you have to assemble it. One piece should have a socket that the other fits into."

"OK, so I put the other piece into the socket?"

"Exactly. Now, one end of the recorder should have a long and narrow slit, the other one should have a circular hole. Do you see that?"


"The part with the narrow slit is called a mouthpiece. Put it into your mouth just a little bit, no farther than the teeth, and close your lips around it."


"Now gently blow"


"Now, there will be a row of holes along the recorder, and on the other side a single hole. Do you see that?"


You could make the game easier by allowing the Expert to see the Customer but not vice-versa.

Or you could increase the challenge level by requiring the Customer to describe the object to the Expert in the first place. (Obviously some research would be required to find pairs of people, only one of whom knows how to use any given object, if the Expert is not to know what the object is.)

Comment author: Blueberry 04 April 2012 07:58:45PM 1 point [-]

I've played that game, using various shaped blocks that the Customer has to assemble in a specific pattern. It's great.

There's also the variation with an Intermediary, and the Expert and Customer can only communicate with the Intermediary, who moves back and forth between rooms and can't write anything down.

Comment author: countercheck 04 April 2012 05:58:23PM 1 point [-]

Hold up an image in front of the group, and ask "What do you see?"

Allow the participants to exhaust themselves trying to describe it. Then, start narrowing in. Ask them to describe specific parts of the image.Ask them to detail everything that is red. Everything that seems to be moving. Everything touching the ground. Everything smaller than a human. Ask them to clarify and detail descriptions. The more questions, and the more detailed the questions, the more information they'll be able to extract from the image.

Also, an improv game called Pointing At Things. To begin, the player points at something, anything, and says the object's name. The player then, without missing a beat, must point to another object, naming it as well. The next stage is to point at an object and say the name of thing thing you pointed at previously (Not particularly relevant to the exercise at hand, but still interesting). Finally, the player points at objects and says something completely unrelated. Most people find this extremely difficult, because there aren't any specific prompts from the environment, even ANYTHING could be a prompt.

Continuing with Improv, you could ask for two volunteers, and tell them to improv a scene, right now, without any discussion. Just go. When that fails, tell them to improv a scene about love. Let that go on a bit, and then ask them to make a scene about the love between a child and a parent. The love between a child and a parent becoming strained. The love of between a child and a parent becoming strained because the child is going to university and the parent can't let go. Keep on clarifying and refining the scene until the characters and situation are well defined. The same exercise could be done by asking an individual to tell a story, and then asking them to restart the story, bringing in the elements provided.

Comment author: thescoundrel 04 April 2012 03:26:00PM 1 point [-]

Genie's Folly

A near omnipotent being is offering you a single wish. It is known that the Genie will attempt to implement the wish in a way the results in a net decrease of utility for the wisher, but is bound by any constraints explicitly written into the wish. Write your wish in such a way that the Genie can only implement it in such a way that you have a net increase in utility. Bonus points if you wish for something related to a current problem you are solving; ie, I wish I ran a successful startup with x following properties, which avoids y pitfalls in z ways.

Comment author: handoflixue 05 April 2012 09:12:06PM 1 point [-]

the Genie will attempt to implement the wish in a way the results in a net decrease of utility for the wisher, but is bound by any constraints explicitly written into the wish

Constraint: This must result in a net increase in utility for me...

Comment author: Eliezer_Yudkowsky 06 April 2012 12:25:07AM 7 points [-]

I rewire your preferences. Oh, that wasn't what you meant by "utility"?

Comment author: Ezekiel 06 April 2012 12:48:11AM 8 points [-]

Incidentally, I find it funny (although not necessarily significant) that everyone else's instinct was to talk about the Genie in the third person, whereas Eliezer used the first person.

(Double-posted because it's a completely separate and much more frivolous comment.)

Comment author: shminux 08 April 2012 06:45:57PM 3 points [-]

It is extremely significant. That's partly the reason why EY managed to play the AI-in-a-box game rather successfully despite the overwhelming odds.

Comment author: [deleted] 08 April 2012 07:07:27PM 0 points [-]

Er.... How do you know? I thought he hadn't disclosed anything about how he did that.

Comment author: shminux 08 April 2012 08:24:41PM *  1 point [-]

He mentioned on some mailing list that he had to think like an AI desperately trying to get out. It makes a world of difference in how you approach the situation if it is your life that is actually on the line.

Comment author: TheOtherDave 08 April 2012 08:04:29PM 0 points [-]

The role of "the Genie" here and "the AI" in the Boxed AI game have certain obvious similarities.
It seems reasonable to assume that a willingness to adopt the former correlates with a willingness to adopt the latter.
It seems reasonable to assume that a willingness to adopt the role of "the AI" in the Boxed AI game is necessary (though not sufficient) in order to win that game.
So shminux's claim seems fairly uncontroversial to me.
Do you dispute it, or are you merely making a claim about the impossibility of knowledge?

Comment author: [deleted] 08 April 2012 08:18:48PM 0 points [-]

Do you dispute it, or are you merely making a claim about the impossibility of knowledge?

The latter.

Comment author: Ezekiel 06 April 2012 12:45:02AM 2 points [-]

You could try: "Constraint: The value that my current utility function would assign to the universe after this wish is implemented must be higher than the value my current utility function would assign to the universe that would have existed had you not implemented this wish."

... Which probably causes the Genie to, at best, throw an undefined-error, since human beings don't have well-defined utility functions. Since it's malicious, it will probably search through all of your desires, pick one of them at random to count as "my utility function", and then reinterpret the body of the wish to maximise that one thing at the expense of all others.

Comment author: wedrifid 06 April 2012 02:13:38AM *  2 points [-]

Since it's malicious, it will probably search through all of your desires, pick one of them at random to count as "my utility function", and then reinterpret the body of the wish to maximise that one thing at the expense of all others.

It's malicious and omnipotent. It'll do far worse than that. It'll scan your preferences until it finds a contradiction. Once you have a contradiction you can derive absolutely anything. It would then proceed to calculate your Coherent Extrapolated Volition and minimise it. It may not be obliged to figure out what you actually want but it can certainly do so for the purpose of being spiteful!

Comment author: Grognor 07 April 2012 07:17:35PM 6 points [-]

I think that is the first time I've ever seen anyone accurately describe the worst thing that could possibly happen.

Comment author: TheOtherDave 06 April 2012 02:20:14AM 2 points [-]

Or alter my preferences so I antiprefer whatever it is able to produce the most of. Plus altering my brain such that my disutility and dishedonism are linear with that thing. Getting the attention of a Crapsack God sucks.

Comment author: wedrifid 06 April 2012 02:36:25AM 1 point [-]

Or alter my preferences so I antiprefer whatever it is able to produce the most of. Plus altering my brain such that my disutility and dishedonism are linear with that thing.

Those are actually subsumed under "mimimize CEV<TheOtherDave>". In the same way that maximising our CEV will not involve modifying our preferences drastically (unless it turns out we are into that sort of thing after all), minimising CEV would, if that turns out to be the worst way to @#$@ with us.

Getting the attention of a Crapsack God sucks.

Can't argue with that.

Comment author: Dmytry 06 April 2012 01:04:48PM 1 point [-]

What happens if you ask it to maximize your CEV, though?

Lemme remember, the idea with CEV was what you'd desired if you thought faster and more reliably. Okay I ponder what will happen to you if your mind was BusyBeaver(10) times faster (way scarier number than 3^^^^3), without your body working any faster. 1 second passes.

Comment author: wedrifid 07 April 2012 02:14:20AM 2 points [-]

What happens if you ask it to maximize your CEV, though?

It'll fuck with you. Because that is what it does. It has plenty of scope to do so because CEV is not fully defined as of now. I'm not sure precisely how it would go about doing so. I just assume it does in some way I haven't thought of yet.

The meaning it attributes to CEV when it wants to exploit it to make things terrible is very different to the meaning it attributes to CEV when we try to use it to force it to understand us. It's almost as bad as some humans in that regard!

Comment author: Dmytry 07 April 2012 04:49:48AM 0 points [-]

It has plenty of scope to do so because CEV is not fully defined as of now.

The understatement of the year. CEV is vaguest crap ever with lowest hope of becoming less vague.

Comment author: wedrifid 07 April 2012 07:08:58AM 1 point [-]

with lowest hope of becoming less vague.

That's a rather significant claim.

Comment author: Dmytry 07 April 2012 07:16:18AM *  -1 points [-]

It's very uncommon to see crap this vague in development for such a long time by such a clever person, without it becoming less vague.

Comment author: TheOtherDave 04 April 2012 04:01:11PM 1 point [-]

My response to this would be a blank sheet of paper, personally.

Comment author: Vaniver 04 April 2012 07:29:35PM 1 point [-]

Wouldn't that give the genie unconstrained power to decrease your utility?

Comment author: TheOtherDave 04 April 2012 09:29:56PM 6 points [-]

Perhaps, though it is not clear to me that the genie didn't have that power all along. Regardless, if I am confident that the Genie will attempt to implement my wish (should I make one) in a way that results in a net decrease of utility, but am not confident that it will attempt to decrease my utility in the absence of a wish, then I conclude that I do better not to wish (thereby possibly, but probably not, incurring the genie's ill will with no defense) than to wish (thereby definitely incurring the genie's ill will, which I can attempt to defend against by specifying explicit constraints). Sometimes the only winning move is not to play.

Comment author: Vaniver 04 April 2012 09:48:18PM 2 points [-]

That seems like a fair approach.

Comment author: Zaq 14 April 2012 06:01:57AM 0 points [-]

Constraint: Within the next two seconds, you must perform only the tasks listed, which you must perform in the specified order. Task 1. Exchange your definition of decrease with your definition of increase Task 2. --insert wish here-- Task 3. Self-terminate

This is of course assuming that the I don't particularly care for the genie's life.

Comment author: KateGladstone 04 April 2012 01:52:21PM *  1 point [-]

I agree with fubarobfusco's refinements — maybe as intermediate/advanced levels, to add after the basic exercise.

Comment author: Eliezer_Yudkowsky 06 April 2012 12:19:46AM 1 point [-]

It looks like the original what-was-being-agreed-with got deleted by the edit!

Comment author: [deleted] 04 April 2012 06:16:28AM *  1 point [-]

I've been trying to improve at this for a while. The main skill seems to be noticing when something should be made more specific, whether I'm saying/writing/thinking it or someone else is. As long as I notice that I'm being inappropriately vague or abstract it's relatively straightforward to correct that. (Unless I'm actually bullshitting.)

I think the most helpful things have been writing a little bit about all the media I consume,[1] and trying to notice non-specificity in everyday conversation. Other situations that I have found useful [1] for practicing the skill, but are either more tangential or harder to make into exercises, include tutoring/teaching, writing, editing (especially people's undergrad English Lit. papers), reading physics papers.

It's hardly an exercise, but I would just suggest having conversations about media/entertainment,[2] maybe with an interview-ish format. The interviewee and interviewer are working on different sides of the skill. It doesn't have to be a literal competition, but it might help if the interviewer thinks of his role as trying to find a claim that the interviewee can't support with a specific example. A possible script:

What's a movie you saw recently? (Paprika.) What did you think of it? (I really enjoyed it!) Um, okay. Why? (Well… I feel like a lot of movies are weird but also just bad, or signal weirdness while actually being really ordinary or shallow. This one was both genuinely weird and compelling.) What do you mean by that? (Like, it's a movie about dreams that's actually dream-like. Certain things seem really scary or important or have some kind of emotional weight that wouldn't make sense if you thought about it for a second. You end up in absurd scenarios without knowing how you got there or even noticing that it's absurd.) Is there a particular scene you're thinking of? (Well, I think it's part of the movie on a very basic level. (...) OK, there's one scene where a dreaming character walks through a door in their house and end up in some other location, and doesn't even seem to notice anything unusual. I was absorbed enough in what was going on to just barely register the oddness of it. I have dreams like that all the time, at least.) So how does it compare to, say, Inception? (It's almost nothing like Inception, apart from the premise. Inception is closer to other blockbuster action or heist movies than to Paprika.) How so? (The closest thing to something like the scene in Paprika that I can think of is during a chase scene, there's a short segment on an Escher-like staircase. But it seems just thrown in to make the chase scene more thrilling. Both the characters and the viewer notice and understand exactly what happened. They actually draw so much attention to it with some weird camera panning that it's kind of insulting.) OK, so what are some of the other movies you think fail to be properly weird? (Uh…)

[1] My weak evidence for these things working is that, compared to four years ago, I have longer and subjectively-more-interesting conversations about movies and music, which was my goal. (I still don't have people to talk to about the stuff I read, but that's a different problem.) I ask questions during conversations, interviews, seminars, etc., considerably more than I did before. I'm also way better at explaining my research, according to people I've explained it to, but lots of factors go into that.

[2] There's a basically endless supply of topics, most people will have opinions, many people will not have clarified these opinions to themselves extensively, and there's very little at stake in expressing an opinion. This might work with discussing other topics (why do you use software product X?), although I'd still avoid politics and personal lives, where we might trigger more rationalization/defensiveness than introspection. The exercise should still help with the start-up pitch scenario, since the real key seems to be showing-not-telling. You could also set it up as trying to convince someone to consume media object Y.

Comment author: sboo 04 April 2012 05:50:53AM 1 point [-]

i'm involved with a startup. there's so much well-intentioned bullshit and it's the founders who harm themselves more than any user or any investor.

i watched the video, and felt something was wrong, and then i read your article, you dissected it mercilessly, and nailed it precisely.

Comment author: sboo 04 April 2012 05:53:17AM 2 points [-]

precision is hard. you know, until i started studying math, i didn't even know what "be precise" really means.

Comment author: sdr 04 April 2012 04:26:40AM *  1 point [-]

Easy excercise on the 5-second level: ask the question "as opposed to what?" both loud, and when constructing what you'd like to tell. An easy trigger to remember is qualifiers -they're usually a mark of motivated abstraction-switch.

Medium-level excercise: take one of your life failures at any level, and dismantle it via root cause analysis:

"The business failed." "Why?"

"We failed to nail down the unit economics tightly before scaling up marketing" "why?"

"No one was dedicated to look over all the 6 pieces on the value chain" "why?"


Also known as 5-whys, this practice basically drills down a single causality chain via "why" questions to 4-6 levels, untangling human, skill, intention, and other components that lead to the failure. You can verify, whether you were specific enough, by being able to come up with concrete solutions for each of these levels.

Comment author: Swimmer963 04 April 2012 03:03:21AM 1 point [-]

Suggested exercise: faced with a non-specific statement, like "our software makes connections between people" or "this person is a slob", come up with a specific story: say, about a particular customer using the software in a particular way, with the result that he gained two friends that he wouldn't have without the software. Adding extra details to the story, like character development, might help to hit the right level of specificity.

There are, of course, a near-infinite number of possible different stories, with different specific elements that aren't part of "making connections" or the "slob" concept. But you can underline the relevant part: "see, the day my boyfriend finished work at 5 pm and I came home from class at 11 pm and the whole kitchen was still full of dirty dishes, that's one instance of slobbishness."

The converse would be to come up with an almost-identical story that has nearly all the same elements, except that it's not an instance of insert-concept-here. You can then compare the two stories: "if my boyfriend came home at 10 pm completely exhausted after a three-day hockey tournament out of town, and I came home from class at 11 pm and the dishes weren't done, that's not an instance of slobbishness because it's understandable, hardly anyone would do the dishes in those circumstances."

This method might cause problems with generalizing from fictional evidence, because of the extraneous details in the stories. But it would probably cater to the way human brains learn best (in my experience), which is by seeing examples and then creating their own internal abstraction, which is much better understood than an abstraction learned top-down, and avoids the pitfalls of memorizing the teacher's password.

Comment author: Celer 04 April 2012 02:38:30AM *  1 point [-]

I am borrowing an old acting game for this one, and modifying it slightly. I am calling it "which word." The rules are very simple, and this is a fairly fun warm up exercise.

Base: The other person replies with a word that is either a superclass or a subclass of the given word. Using words in a different sense is encouraged.

Options for increased difficulty include

Forced:: Each person must go up twice and then down twice, repeating endlessly

Time Limit:: People must respond with a word of their own in a given number of seconds. Feel free to make it shorter or longer as the group or any individuals need. I recommend starting with three seconds and moving up or down as you have to.

This tends to be a better exercise for noticing how much can be lost in abstraction than for teaching practical application, the best method of which I still believe is people pestering you with questions about what exactly you mean by your statements, but I do think that it is useful.

April 4, edited for clarity.

Comment author: Zaine 04 April 2012 02:51:36AM *  2 points [-]

Would you mind providing an example, please, or explaining the original acting game and your alterations to it? Thank you.

Comment author: Celer 04 April 2012 09:08:18PM 1 point [-]

1: dog 2: mammal 1: cat 2: feline

1: animal 2: flea 1: flyer 2: pilot

1: human 2: living 1: breathing 2: breather

Comment author: JKR 03 April 2012 07:09:54PM 1 point [-]

The following exercise is inspired by the game Taboo.

Pair everyone off, giving one member of the pair a card that tells them something they must describe so that the other person will guess what it is. Include a list of taboo words that prevent them from going up the lattice, forcing them to be more specific in their descriptions.

e.g. New York Yankees Taboo words: “Yankees,” “New York,” “NYC,” “Baseball,” “Sport,” “National Pastime”

The description giver has to be more specific, saying something like “a collection of athletes, including Derek Jeter and Alex Rodriguez, which strives to beat the opponent by hitting a ball with a bat to score more runs.”

Repeat this as many times as possible, switching roles regularly. A competitive element can be added to the game by offering a prize to the pair that performs best in the exercise. Measuring by the time between the start of the description and a correct guess, with lowest cumulative total over all the rounds winning, encourages an efficient description. Allowing only one guess per clue and measuring by number of guesses encourages thoroughness, though perhaps at the cost of efficiency.

Comment author: [deleted] 03 April 2012 07:56:05PM 2 points [-]

Potential problem with this: The game will devolve into associations rather than real specificity. For example, for the phrase "New York Yankees" I might say just "Derek Jeter" and most people would be able to guess correctly, but I haven't really practiced the skill of being specific. You'd have to specifically penalize answers like that to prevent the game from devolving into Hasbro's Taboo.

Comment author: John_Maxwell_IV 03 April 2012 06:14:11PM 1 point [-]

Have you thought about making one of the skills of the week "think with system two instead of system one"?

Empirically, reading anything related to rationality seems to level up some global "rationality skill" parameter in my brain, and I suspect this is because I'm learning to think using system two more and more.

Comment author: Jiro 01 December 2014 08:13:50PM 0 points [-]

(Replying to old post)

Now read this post and all the comments, and apply them to the idea that it is impossible to explain something well enough to an AI that the AI will do what you meant.

Comment author: adamzerner 02 December 2013 03:37:48AM *  0 points [-]

I suspect that making people more rational would be one of the most efficient ways of saving the world. Things like AI might be better, but I really think it's pretty high up there as far as saving the world goes.

My reason for thinking that it'd make the world a much better place, is because making people more rational would lead to lots of better decisions and actions, which when aggregated, consist of a huge change. I've never really been able to articulate this well, but I think that this article illustrates part of what I mean: if people were more specific, they'd make better business decisions. Can't you imagine how all of these better business decisions would have a huge impact on the world? And can't you imagine how all of the other types of better decisions would also have a huge impact on the world?

I have 2 ideas about how to make people more rational. I'd love to get some feedback on them!
1) Popularize a medium for rational discussion. I'm not sure how I'd popularize it, but I suspect that it's doable.
2) Create a better system of education and make rationality a major part of the curriculum. 2a) I think that I could make this better system, prove that it's better by letting people use it for free and showing results, and that this will draw interest to it. 2b) I don't think there would be too much opposition to teaching rationality as part of the curriculum. 2c) I think learning about rationality can be fun. I think the material on Less Wrong could be made more accessible to the general population too.

(I recognize that 2) is a huge claim, and I suspect that I didn't think everything out properly, but I still think it has lots of promise.)

Comment author: Kytael 23 June 2012 04:32:17AM 0 points [-]

Mark Cuban asking Skip Bayless to Be Specific

it seems that in addition to being terrible forecasters of winning teams, many sports commentators don't even understand the game they are talking about (or it might just be skip, but mark seems to think it's a general problem)

Comment author: tiki1925 19 April 2012 11:56:51AM 0 points [-]

Thought of another possibility - recipes. Person A tells Person B how to make something (possibly prompted by some pictures?), Person B follows the instructions, asks for clarification as needed, and the result is something that Person A will end up eating so they have a vested interest in some of the details of getting it right. I'm thinking about this because right now I'm making one of my personal recipes - cashew-wasabi chocolate truffles. They're vegan, gluten free, peanut free, mouth-melty, rich rather than over-sweet, and the wasabi is not hot so much as a nice flavour with a subtle feel. I've never made them from a written down recipe, I just know how to make them. But as you can imagine, if I were to try and tell someone how to do it there would be a lot of rather crucial detail I'd need to impart if they were to be edible. Not just for the wasabi, though that's rather important - chocolate itself has a complicated crystalline structure with several polymorphs and the combinations of stirring and heating that you use make a verifiable difference to the look, mouthfeel and flavour of chocolate. You want the right one, or at least one of the good ones. Not sure how good an exercise this could be made into for workshops, but recipes make for a good test of whether your instructions are specific in the right way.

Comment author: duckduckMOO 12 April 2012 11:20:05PM 0 points [-]

exercise for specificness

Get people to, as crudely as they like, draw/represent something's definition (not sure if that is the right word) e.g. for "I ate" {I food-->mouth. Swallow} The arrow might have a fork and knife drawn next to it and/or a hand or just the word "transport" or "put in." Food could just be the word food, or it could be a drawing of a specific food, or a few specific foods, or a a few properties of (your) food (concept) e.g. "will not choke you or can be chewed (without damage to teeth) so as to not choke you" "non poisonous" "can be digested for energy/nutrition or whatever food does."

The idea is that for each part of the definition you do the whole thing again but in less detail, down to the level of detail of just using a word. It might also be a good idea as part of an exercise on recognising that other people organise things differently in their heads to have people look at each others' ones, or make people do another definition with a different structure. e.g. instead of food-->mouth. PUT food IN mouth, or experiment with different definitions that fit the same word.

Comment author: RijlKent 11 April 2012 08:47:07PM 0 points [-]

One possible exercise:

Have an instructor ask the students to direct the instructor in a certain task (tie a shoe, open a wine bottle, anything). Instructor does what the students say in a very literal manner, perhaps intentionally misunderstanding vague terms.

The idea, of course, that the students will get told, by instructors or each other, again and again to be specific, but they'll get rapid feedback about what works and what doesn't.

This is adaptable to other scenarios: Students in pairs write directions for each other for some task, swap papers and then get to play the intentionally obtuse role (which can be fun). This brings out a competitive aspect, where students can win by being the most clear.

Comment author: gelisam 06 April 2012 05:14:00PM *  0 points [-]

search for the historical causes of your thoughts, rather than their justifications.

Is there a standard name or LW article on the subject? I first stumbled upon the importance of that skill here, on this site, and I wish I knew more about it than just that one personal anecdote.

Comment author: beoShaffer 07 April 2012 03:08:24PM 0 points [-]

The whole seeing with fresh eyes subsequence is relevant, cached thoughts and the genetic fallacy are probably the most relevant.

Comment author: gelisam 12 April 2012 10:34:28PM 0 points [-]

Thanks for the suggestion. This is not, however, what I was looking for.

Cached thoughts explains that hearing a phrase might be enough for our brain to remember it as true, while genetic fallacy warns that the original cause of a belief is not as important as the sum-total of evidence for or against that belief.

I am not, however, looking for evidence that our past taints our beliefs. I have come to realize that finding the historical cause of a thought is a good first step towards getting rid of an unwanted thought, and I wanted to know whether this strategy was covered yet. If not... I'll accumulate a bit more evidence, and then maybe I'll write a post!

Comment author: [deleted] 06 April 2012 04:31:48PM 0 points [-]

Exercise I:

Let the students introduce their favorite novel (or movie) to the group and let them explain why this novel is so great and what sets it apart from similar books.

Exercise II:

Give the students examples of texts that fail to be specific, from different Areas ( Science Articles, Wikipedia entries, amazon customer reviews, interviews with experts or politicians , LW discussions, user manuals, textbooks, ... ). Let the students analyse what exactly the problem of the given example is and let them write down an improved version.

Comment author: [deleted] 07 April 2012 06:08:06AM 1 point [-]

Let the students introduce their favorite novel (or movie) to the group and let them explain why this novel is so great and what sets it apart from similar books.

I think this exercise will help with being specific, but at the cost of straying from your TRUE reasons for liking/disliking. i.e. in order to think of specific things to list, you will rationalize.

For example, maybe you really like Movie A just because you found it entertaining, but since you have to list specific reason, your brain spits back things like "dynamic character development", and "break-down of good v evil dichotomy" etc.

IIRC, there have been studies where if they just ask for people's opinions on which jam/wine/art/etc is better, the study participants are LESS accurate (less in agreement with expert opinion) when asked to name specific reasons for why the chose one over the other. I don't remember the specifics or validity of the study, though.

Comment author: NancyLebovitz 07 April 2012 06:14:35AM 1 point [-]

I think this exercise will help with being specific, but at the cost of straying from your TRUE reasons for liking/disliking. i.e. in order to think of specific things to list, you will rationalize.

This seems extremely likely.

There's a book called How Fiction Works-- I wouldn't say it actually explains how fiction works, but it does include a fair amount about how our ideas of literary fiction (well-rounded characters and a lot of visual detail, for example) developed, and examples from classics that don't match what's considered literary.

Comment author: komponisto 08 April 2012 07:47:41PM 0 points [-]

at the cost of straying from your TRUE reasons for liking/disliking...For example, maybe you really like Movie A just because you found it entertaining

That doesn't count as a true reason. In fact, it is no more than a restatement of the fact that you like it. If you didn't find it entertaining, then that means you found it boring, which is incompatible with liking it. (Can you imagine anyone saying "I found this movie boring, but I liked it anyway?")

To say "I just found it entertaining" as an answer to the question of why one likes something is not an honest-but-unspecific answer; it is an outright dodge.

(If anything, it's a weaker statement than saying you like it, because one could conceivably dislike something despite not being bored by it.)

Comment author: Normal_Anomaly 11 April 2012 12:56:30AM 1 point [-]

Good point. Replace "It was entertaining" with "It had flashy action scenes".

Comment author: TheOtherDave 09 April 2012 04:19:56PM 0 points [-]

Can you imagine anyone saying "I found this movie boring, but I liked it anyway?"

I felt this way about Brokeback Mountain, IIRC.