Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

The Featherless Biped

1 Post author: Annoyance 02 September 2009 05:47PM

The classical understanding of categories centers on necessary and sufficient properties.  If a thing has X, Y, and Z, we say that it belongs to class A; if it lacks them, we say that it does not.  This is the model of how humans construct and recognize categories that philosophers have held since the days of Aristotle.

Cognitive scientists found that the reality isn't that simple.

Human categorization is not a neat and precise process.  When asked to explain the necessary features of, say, a bird, people cannot.  When confronted with collections of stimuli and asked to determine which represent examples of 'birds', people find it easy to accept or reject things that have all or none of the properties they associate with that concept; when shown entities that share some but not all of the critical properties, people spend much more time trying to decide, and their decisions are tentative.   Their responses simply aren't compatible with binary models.

Concepts are associational structures.  They do not divide the world clearly into two parts.  Not all of their features are logically necessary.  The recognition of features produces an activation, the strength of which depends not only on the degree to which the feature is present but a weighting factor.  When the sum of the activations crosses a threshold, the concept becomes active and the stimulus is said to belong to that category.  The stronger the total activation, the more clearly the stimulus can be said to embody the concept.

Does this sound familiar?  It should for us - we have the benefit of hindsight.  We can recognize that pattern - it's how neural networks function.  Or to put it another way, it's how neurons work.

But wait!  There's more!

Try applying that model to virtually every empirical fact we've acquired regarding how people produce their conclusions.  For example, our beliefs about how seriously we should take a hypothetical problem scenario depend not on a rigorous statistical analysis, but a combination of how vividly we feel about the scenario and how frequently it appears in our memory.  People are convinced not only by the logical structure of an argument but the traits of the entities presenting it and the specific way in which the arguments are made.  And so on, and so forth.

Most human behavior derives directly from the behavior of the associational structures in our minds.

To put it another way:  what we call 'thinking' doesn't involve rational thought.  It's *feeling*.  People ponder an issue, then respond in the way that they feel stands out the most from the sea of associations.

Consider the implications for a while.

Comments (63)

Comment author: RichardKennaway 03 September 2009 09:47:03AM *  5 points [-]

Does this sound familiar?

Yes. I see nothing here not already covered by this, this, and this.

Your final conclusion is like saying that the computation done by computers doesn't involve arithmetic. It's *flow of electric charge*. The charge flows around, then settles down in some stable point in the sea of possible distributions. ETA: On that point, see also this.

Comment author: Annoyance 04 September 2009 07:54:54PM -1 points [-]

Your final conclusion is like saying that [blah blah blah]

No, it's not. Associational processing can emulate logical thinking, but it's not restricted to it and will not normally produce it. Restrictions have to be added for logic to arise out of the sea of associations.

Comment author: RichardKennaway 04 September 2009 09:17:09PM 1 point [-]

No, it's not. Associational processing can emulate logical thinking, but it's not restricted to it and will not normally produce it. Restrictions have to be added for logic to arise out of the sea of associations.

In other words, we have to learn logic, we're not born with it. This is news?

Electric charge doesn't spontaneously do arithmetic either.

Comment author: byrnema 03 September 2009 03:33:40AM *  5 points [-]

People are convinced not only by the logical structure of an argument but the traits of the entities presenting it and the specific way in which the arguments are made.

[…]

Consider the implications for a while.

I know that the main stance here on LW is that we need to improve our thinking, and I completely agree with that, but in this comment I would like to make the following point: as humans, we actually are really good at thinking already. We’ve evolved a certain way of thinking, and it makes sense to think about why we think that way and why it works so well to think that way.

The way thought actually goes (before it is tamed via talking and writing) seems chaotic, undisciplined and opaque (or instead: organic, flexible and efficient). We have a model that transparent, linear logical thought is superior and more reliable, but I suspect this has more to do with the effectiveness of our communication than our thought process. (People who harness transparent, linear logical thought are better at communicating their ideas accurately.)

Whenever I think about something, I notice that a heuristic explanation will arrive first and that this heuristic answer is more reliable than any logical linear answer I subsequently come up with. Therefore, heuristic solution in hand, communication of ideas to myself or others becomes a problem of finding a linear logic that roughly fits the networked solution. (This could be criticized as trying to find an argument that fits the conclusion, but I think this is uncharitable. It is the uncertain process of approximating a complex multi-dimensional organic network with a static, linear causal chain.)

On the one hand, I think a networked, heuristic, gut answer should be re-expressed in a logical linear way, for the purposes of error-checking, fine-honing and communication, and you almost always gain deeper understanding from doing so. (E.g., the network gains clarity and faulty or irrelevant links are down-voted). If there is a discrepancy between a linear argument answer and a heuristic answer, you can compare the two, trying to determine if there is something important missing from the linear argument or a false connection in the heuristic argument.

On the other hand, in the case of a conflict between a heuristic answer and a logical linear answer, I will always go with the heuristic. I realize that this probably sounds like blasphemy on a rationality site, but I am being honest and I can justify this position. (To myself, heuristically – we will see if I can do so linearly.)

Suppose that reasoning logically one way gives an answer, but my gut feeling is that the answer is something else. Experience has taught me that the heuristic always wins with understanding the correct answer first. But this makes sense. If the heuristic is not convinced by a logical argument, then either the logical linear argument is wrong or I haven’t fully understood it, in which case I’m not qualified to have confidence in it. As soon as I have understood the argument, and if I agree with it, my heuristic incorporates the argument. Thus by the time I have understood a correct logical argument, the logical reasoning is just a subset of my functioning heuristic solution.

So I don’t think it’s logical to prefer a logical argument to a heuristic argument, but of course still, a heuristic with a logical backbone is much more solid than one without.

Comment author: billswift 03 September 2009 04:41:19PM 3 points [-]

I have written a bit about the relation between logical and heuristic thinking and I think you did an excellent comment; you might consider expanding it slightly as a top level post.

On the other hand, in the case of a conflict between a heuristic answer and a logical linear answer, I will always go with the heuristic.

Unless time limited, I usually try to find the error in my thinking. Usually, it will be some factor I missed in my logical analysis, but sometimes it will be some factor my gut feeling didn't accurately weigh. That is why we need to continue learning about availability heuristic and other biases, we need to learn them so thoroughly that our gut feelings take them into account.

Comment author: thomblake 03 September 2009 12:41:03PM 0 points [-]

This could be criticized as trying to find an argument that fits the conclusion, but I think this is uncharitable.

I think it's exactly right. All reasons are rationalizations.

Comment author: Annoyance 04 September 2009 08:00:39PM 1 point [-]

Not in the way that 'rationalization' is used in natural language. That refers to a non-rational statement that is used in place of rationality in order to satisfy the desire to present an argument as rational without having to go through the trouble of actually constructing and adopting a rational position.

The biggest functional difference: when a reason is abolished, the behavior goes away. When a rationalization is abolished, the behavior remains.

Comment author: anonym 03 September 2009 03:47:21AM 2 points [-]

Wittgenstein was a philosopher who described the inadequacies of "necessary and sufficient conditions" for concepts/categories long before cognitive science existed.

Maybe it's not quite as simple as "philosophers bad, cognitive scientists good"?

Comment author: thomblake 03 September 2009 01:38:44PM 0 points [-]

By Wittgenstein's time, there were already plenty of philosophers who thought definitions aren't quite captured by necessary and sufficient conditions.

Comment author: Annoyance 04 September 2009 07:57:15PM -2 points [-]

And the recognition that the process that ordinary people went though had pretty much NOTHING in common with "necessary and sufficient conditions" was not made by philosophers.

Ordinary people struggle to decide whether dolphins are fish or penguins are birds. And they often get it wrong if they haven't been explicitly taught otherwise; even then, some still screw up their answers.

Comment author: Technologos 05 September 2009 06:05:48AM 1 point [-]

At what point do we say that the problem lies in the definition of a category? Since ordinary people have no especial use for the category "bird," it's unsurprising that they haven't nailed down characteristics that would allow such a use.

Categories that we need--that must reliably possess some characteristic(s) such that they are useful--tend to have strict necessary and sufficient conditions for inclusion. Categories that we use purely to simplify speech can get away with fuzzier definitions.

Is the dolphin really a fish? That depends: is that thing over there really a blegg?

Comment author: Annoyance 08 September 2009 01:22:39PM -1 points [-]

The biological category of 'mammal' is quite well-defined, thank you.

And fuzzy definitions are fine until you're dealing with a case that lies in the penumbra, at which time it becomes a massive problem.

Comment author: Technologos 08 September 2009 04:23:39PM -1 points [-]

If a fuzzy definition becomes a massive problem, then that definition clearly wasn't in existence merely to simplify speech.

Regarding mammals, is there a use for the term that requires its inclusion of dolphins? Does the existence of sweat glands usefully separate mammals from other animals? After all, mammals in general share a variety of properties: most give live birth, most have hair, most are warm-blooded, etc.--but we admit to the category of mammals many animals that fail one or more of these criteria.

A well-defined but useless category (I am not arguing that "mammal" is such a category, as there may well be a biological use for it) may be pedagogically interesting but otherwise may merely confuse our understanding of thingspace.

Comment author: Annoyance 31 October 2009 01:51:52PM -2 points [-]

".--but we admit to the category of mammals many animals that fail one or more of these criteria."

No, we don't. Dolphins have all of the required attributes to be considered mammals. If they didn't, we couldn't call them mammals any longer.

Comment author: DanArmak 31 October 2009 02:06:53PM *  3 points [-]

The definition of a mammal is simple: descent from the most recent common ancestor of all mammals. In practice, to avoid circularity, it is sufficient to take the MRCA of a few indisputable mammalian groups such as primates, rodents, carnivorans, ungulates, etc. to include all mammals.

This definition is useful because it turns out that there are many traits unique to mammals, and any given mammal will have almost all these traits. Many such traits are anatomical/biochemical/etc. (Many outwards traits like live birth or so-called "warm blood" aren't unique to mammals.)

However, even if this definition wasn't useful to us, the group Mammalia would still exist. It's a natural evolutionary group (clade) in phylogenetics, to which we merely give a name. (Edit: and cladistics is a natural way of classifying species (among other ways). By natural I mean a classification that tends to match common and unique traits of species in the same clade, and which is causally linked to the history of of the species and to predictions for their future, so that I would expect aliens to have a relatively high probability of using similar classifications.)

The precise clade referred to by the word Mammalia can change depending on context. It makes sense to ask whether borderline species like platypuses are mammals or a sister group of mammals. That's the fuzzy nature of any classification of real things. But the natural limits of the category "mammals" lie somewhere around the monotremes. A group which doesn't include dolphins is definitely not the group of all mammals.

Comment author: AllanCrossman 31 October 2009 07:45:38PM *  2 points [-]

to avoid circularity, it is sufficient to take the MRCA of a few indisputable mammalian groups such as primates, rodents, carnivores, ungulates, etc. to include all mammals

But the MRCA of "indisputable" groups won't be an ancestor of basal groups like the monotremes or marsupials.

However, there's no dispute about including monotremes. The clade that excludes them is called the Theria. Likewise with the marsupials: the clade that excludes both them and the monotremes is the Eutheria. Every clade potentially has a name; Mammalia is just a particularly well known one.

Things get dicey if the evolutionary relationships are unclear, of course, or if some conventional group is recognised as not being a true clade.

Comment author: DanArmak 31 October 2009 08:14:16PM 2 points [-]

You're right, of course. I was just pointing out that clades nest nicely. Whether you talk about Theria or Eutheria, the species included or excluded by the differences will be the most distantly related ones such as monotremes; but no clade anywhere similar in scope to Eutheria would be able to exclude dolphins. In that sense, it doesn't matter much which "indisputably" mammalian groups you take, their MRCA will be an ancestors of dolphins as well. For instance, the MRCA of humans and of cats is also an ancestor of dolphins.

Comment author: wedrifid 31 October 2009 04:29:29PM *  1 point [-]

a few indisputable mammalian groups such as [...] carnivores

carnivore- any animal that feeds on flesh; "Tyrannosaurus Rex was a large carnivore"

Comment author: Alicorn 31 October 2009 06:24:01PM 2 points [-]

I suspect that "carnivore" there was meant to indicate "the order Carnivora".

Comment author: Technologos 31 October 2009 08:49:27PM *  -1 points [-]

Monotremes are mammals, yet do not share the property of giving live birth.

"Hair... may be greatly reduced in the Cetacea (i.e. dolphins), where it is found as a few scattered bristles about the lips or often present only in the young." W. J. Hamilton, American Mammals.

Heterocephalus glaber, the naked mole rat, is a mammal despite being poikilothermic.

As DanArmak notes, mammals are explicitly not the set of all species fulfilling a particular set of (external) criteria. They are defined by descent.

To the extent that this is useful, great. My point was merely that there is no external fact of the matter that requires drawing the boundaries where they currently are.

So when “ordinary people…get it wrong” by believing that dolphins are fish, it’s a little hard to blame them.

Comment author: DanArmak 31 October 2009 09:33:19PM 0 points [-]

Monotremes are mammals, yet do not share the property of giving live birth.

And many decidedly non-mammal animals do give live birth. Wikipedia lists "scorpions, some sharks, some snakes, [...] velvet worms [and] certain lizards".

Comment author: DanArmak 31 October 2009 09:28:32PM *  0 points [-]

What exactly do you mean when you say they believe that dolphins are fish? The question isn't about what to call a fish.

If they believe dolphins are like tuna in that they live entirely in the sea, have no hands and feet, swim with a thrashing tail, and eat other fish, then they are right.

And if they believe dolphins are like tuna in that they breath through gills and lay single-cell eggs into the water which grow into baby fish, then they are wrong. And I'd happily blame them for being wrong, because these are things that would be easy to get right with a little observation.

Comment author: Technologos 31 October 2009 09:41:51PM 1 point [-]

I'm suggesting that people see dolphins sharing the (easily-observable) characteristics they do with tuna and aren't especially interested in the respiratory or reproductive aspects--particularly because the category of "air-breathing fish" is wrong in some sense by construction.

They are also wrong on the biochemical differences, but insofar as fish are pretty much just "all the animals in the sea" to the general public, I'd say the scientific and the popular use of the word probably diverge.

Perhaps I'm being too much of an economist, but I figure that the actual need for most people to know most of these biological details is limited, and the fuzziness in the popular use of the word arises from a low-effort sweep of the observables.

Comment author: Larks 02 September 2009 11:33:34PM 2 points [-]

When confronted with collections of stimuli and asked to determine which represent examples of 'birds', people find it easy to accept or reject things that have all or none of the properties they associate with that concept; when shown entities that share some but not all of the critical properties, people spend much more time trying to decide, and their decisions are tentative. Their responses simply aren't compatible with binary models.

In a natural environment, people's uncertainty could be uncertainty about their knowledge of the entity (does that thing really have cloven hooves? does that flying thing have feathers?), rather than about the concept (does a kosher beast have to have cloven hooves? does a bird have to have feathers?). It's possible that people's uncertainty in conditions where they are told that the beast has such and such characteristics is due to their methods of reasoning not being developed for such situations, which are rare in real life.

Comment author: SilasBarta 02 September 2009 09:45:41PM *  2 points [-]

Voted up, despite the lack of links to related material, because I think it's an important and far underappreciated point. Both in your life, and in AI design, you need to think, "How did this choice/event/phenomenon even come to my attention in the first place?"

(ETA: Hegemonicon made a largely similar point, with good citations.)

For an example of this oversight in action, refer to my previous qualified criticism of an AI lab's automated scientist, where it's easy to miss how much of the "attention focusing" the team did before their program even saw what was left of the problem.

I would revise this though:

what we call 'thinking' doesn't involve rational thought. It's feeling. People ponder an issue, then respond in the way that they feel stands out the most from the sea of associations.

What we call thinking doesn't necessarily involve rational thought, but you can readjust your thinking processes to better align with rationality. Indeed, that's the whole point of this site.

Comment author: thomblake 02 September 2009 07:28:41PM *  3 points [-]

This post could use some citations.

Comment author: MichaelBishop 02 September 2009 09:42:46PM 2 points [-]

Yes, and the last bit needs more explanation/elaboration. What are the non-obvious implications?

Comment author: [deleted] 03 September 2009 05:38:56AM 0 points [-]

To go on a bit of a tangent...

When asked to explain the necessary features of, say, a bird, people cannot.

Upon seeing this, I decided to try to come up with my best guess as to what the necessary features of a bird are. So I came up with the birdiest non-bird I could think of, thinking qualitatively: why is a bat not a bird? I realized that bats don't lay eggs, but platypuses, a mammal, do; since I couldn't think of a mammal that both lays eggs and flies, but I could think of a mammal that does each, I postulated a hypothetical mammal that does lay eggs and fly, and compared it to a hypothetical bird with fine, hair-like feathers, and pondered what would distinguish these; out of somewhere, the idea came to me that birds' lungs are different somehow. So that was one of my distinguishing features: birds have a certain type of lung.

I also considered the birdiest non-birds I could think of, thinking taxonomically; these were the reptiles. It was easier to come up with a distinguishing feature here: birds are warm-blooded.

So, my thought left me with this best definition: "A bird is a warm-blooded organism with a certain type of lung." Looking up on Wikipedia what type of lung birds have, I was able to refine this into this: "A bird is a warm-blooded organism with circulatory lungs." How close did I come?

Comment author: scotherns 03 September 2009 08:05:36AM 1 point [-]

A bird is a warm-blooded organism with circulatory lungs." How close did I come?

So if I removed the lungs of chicken, you would no longer consider it a bird? Or if I surgically modified some other creature (e.g. a pig) to have circulatory lungs, you would consider this to be a bird?

This kind of argument is why it is pretty difficult to come up with a comprehensive set of features for a broad category like 'bird'. Often the best you can do is produce a set of examples demonstrating the category. Humans are pretty good at such pattern recognition from a set of data.

Like a lot of things, it is hard to define, but you know it when you see it :-)

Comment author: cousin_it 03 September 2009 09:05:26AM *  3 points [-]

So if I removed the lungs of chicken, you would no longer consider it a bird? Or if I surgically modified some other creature (e.g. a pig) to have circulatory lungs, you would consider this to be a bird?

Different people's concepts of "bird" agree on most real-world examples, but I see no reason why they should agree on all conceivable hypothetical examples, so the task of "defining" a word is futile.

Warrigal gave a good recognition algorithm: it inspects a small subset of properties and gives an answer that accords with our judgment in most real-world cases. That's about as far as one can or should go when "defining" something outside of mathematics.

Comment author: Nubulous 03 September 2009 09:40:09AM 0 points [-]

Warrigal gave a good recognition algorithm

Even though no bird, in the history of the world, has ever been recognised using it ?

Comment author: cousin_it 03 September 2009 10:37:56AM *  6 points [-]

When someone proposes a new algorithm, "this algorithm has never been used" doesn't sound to me like a valid critique. More substantively, Cuvier proposed similar outlandish-sounding algorithms tuned to recognizing animals by teeth and bone fragments, which have enjoyed widespread use ever since.

A small anecdote: one of Cuvier's students once dressed in a devil's costume and entered his room at night to scare him. Cuvier opened his eyes, said "Horns? Hooves? You can't eat me, you're a herbivore" and went back to sleep.

Comment author: CronoDAS 03 September 2009 03:27:15PM 0 points [-]

"Horns? Hooves? You can't eat me, you're a herbivore"

Heh.

Plenty of herbivores can still do serious damage when they're annoyed, though. ;)

Comment author: [deleted] 03 September 2009 09:02:23AM 1 point [-]

Well, if you remove the lungs of a chicken, its species is still a species whose members typically have circulatory lungs. What I was wondering is whether there are any species of bird that are not warm-blooded with circulatory lungs, or if there are any species of anything else that are.

Comment author: CronoDAS 03 September 2009 05:52:52AM *  1 point [-]

Taxonomists have a pretty ugly job, having to fit messy real-world objects into categories and having those categories be sharply defined.

For example, do you want to put mushrooms and other fungi into the category "plant"? It seems like a plant, has a cell wall like a plant, is mostly immobile like a plant, but it doesn't perform photosynthesis!

Comment author: [deleted] 03 September 2009 07:02:29AM -1 points [-]

I only said "taxonomically" instead of "phylogenically" because I couldn't think of the latter word. I would imagine a phylogenist's job to be a bit easier: fungi are more closely related to animals than to green plants; therefore, unless animals are plants, fungi are not plants. I'd say it's pretty darn convenient for the phylogenist that, at least in eukaryotes, everything can be organized into neat, sharply defined categories.

Comment author: alexflint 29 September 2009 08:49:24AM 1 point [-]

The recognition of features produces an activation, the strength of which depends not only on the degree to which the feature is present but a weighting factor. When the sum of the activations crosses a threshold, the concept becomes active and the stimulus is said to belong to that category.

This is also how linear classifiers in machine learning work, and many other statistical classifiers just replace "sum" with "something else" (support vector machines etc). On pattern recognition problems like "does this image contain a tree?" or "will this person return their loan?" they far outperform human-tuned decision trees, which classify by asking a series of yes/no questions. That's the nature of the complex sensory information we have to process, and it's not surprising that our brains work like that.

Comment author: byrnema 02 September 2009 08:55:36PM *  1 point [-]

I voted this up because the associations I had while reading this post were quite vivid.

I've just recently started thinking about the way I think in the way described in this post. Whenever I think about something, there is indeed an activated network of interconnected ideas. Some observations:

  • Certain links and nodes are brighter than others due to how frequently I've thought about them before or how much interest I have in them.

  • If I understand something well, the links and nodes are easily accessible and don't change very much as I follow them.

  • If I don't understand something well, the network keeps shifting and changing while I interact with it.

  • Regardless of what I am thinking about, the network is larger than my immediate focus and most of it is vague but I can inspect one link or group of links at a time.

Comment author: fburnaby 03 September 2009 03:11:41PM 0 points [-]

Mathematicians have tried to find ways of dealing with this sort of thing, too: http://en.wikipedia.org/wiki/Fuzzy_set

Do you think this method of modelling would make the problem soluble? Or are there still issues?