I have watched more than one conversation—even conversations supposedly about cognitive science—go the route of disputing over definitions. Taking the classic example to be "If a tree falls in a forest, and no one hears it, does it make a sound?", the dispute often follows a course like this:
If a tree falls in the forest, and no one hears it, does it make a sound?
Albert: "Of course it does. What kind of silly question is that? Every time I've listened to a tree fall, it made a sound, so I'll guess that other trees falling also make sounds. I don't believe the world changes around when I'm not looking."
Barry: "Wait a minute. If no one hears it, how can it be a sound?"
In this example, Barry is arguing with Albert because of a genuinely different intuition about what constitutes a sound. But there's more than one way the Standard Dispute can start. Barry could have a motive for rejecting Albert's conclusion. Or Barry could be a skeptic who, upon hearing Albert's argument, reflexively scrutinized it for possible logical flaws; and then, on finding a counterargument, automatically accepted it without applying a second layer of search for a counter-counterargument; thereby arguing himself into the opposite position. This doesn't require that Barry's prior intuition—the intuition Barry would have had, if we'd asked him before Albert spoke—have differed from Albert's.
Well, if Barry didn't have a differing intuition before, he sure has one now.
Albert: "What do you mean, there's no sound? The tree's roots snap, the trunk comes crashing down and hits the ground. This generates vibrations that travel through the ground and the air. That's where the energy of the fall goes, into heat and sound. Are you saying that if people leave the forest, the tree violates conservation of energy?"
Barry: "But no one hears anything. If there are no humans in the forest, or, for the sake of argument, anything else with a complex nervous system capable of 'hearing', then no one hears a sound."
Albert and Barry recruit arguments that feel like support for their respective positions, describing in more detail the thoughts that caused their "sound"-detectors to fire or stay silent. But so far the conversation has still focused on the forest, rather than definitions. And note that they don't actually disagree on anything that happens in the forest.
Albert: "This is the dumbest argument I've ever been in. You're a niddlewicking fallumphing pickleplumber."
Barry: "Yeah? Well, you look like your face caught on fire and someone put it out with a shovel."
Insult has been proffered and accepted; now neither party can back down without losing face. Technically, this isn't part of the argument, as rationalists account such things; but it's such an important part of the Standard Dispute that I'm including it anyway.
Albert: "The tree produces acoustic vibrations. By definition, that is a sound."
Barry: "No one hears anything. By definition, that is not a sound."
The argument starts shifting to focus on definitions. Whenever you feel tempted to say the words "by definition" in an argument that is not literally about pure mathematics, remember that anything which is true "by definition" is true in all possible worlds, and so observing its truth can never constrain which world you live in.
Albert: "My computer's microphone can record a sound without anyone being around to hear it, store it as a file, and it's called a 'sound file'. And what's stored in the file is the pattern of vibrations in air, not the pattern of neural firings in anyone's brain. 'Sound' means a pattern of vibrations."
Albert deploys an argument that feels like support for the word "sound" having a particular meaning. This is a different kind of question from whether acoustic vibrations take place in a forest—but the shift usually passes unnoticed.
Barry: "Oh, yeah? Let's just see if the dictionary agrees with you."
There's a lot of things I could be curious about in the falling-tree scenario. I could go into the forest and look at trees, or learn how to derive the wave equation for changes of air pressure, or examine the anatomy of an ear, or study the neuroanatomy of the auditory cortex. Instead of doing any of these things, I am to consult a dictionary, apparently. Why? Are the editors of the dictionary expert botanists, expert physicists, expert neuroscientists? Looking in an encyclopedia might make sense, but why a dictionary?
Albert: "Hah! Definition 2c in Merriam-Webster: 'Sound: Mechanical radiant energy that is transmitted by longitudinal pressure waves in a material medium (as air).'"
Barry: "Hah! Definition 2b in Merriam-Webster: 'Sound: The sensation perceived by the sense of hearing.'"
Albert and Barry, chorus: "Consarned dictionary! This doesn't help at all!"
Dictionary editors are historians of usage, not legislators of language. Dictionary editors find words in current usage, then write down the words next to (a small part of) what people seem to mean by them. If there's more than one usage, the editors write down more than one definition.
Albert: "Look, suppose that I left a microphone in the forest and recorded the pattern of the acoustic vibrations of the tree falling. If I played that back to someone, they'd call it a 'sound'! That's the common usage! Don't go around making up your own wacky definitions!"
Barry: "One, I can define a word any way I like so long as I use it consistently. Two, the meaning I gave was in the dictionary. Three, who gave you the right to decide what is or isn't common usage?"
There's quite a lot of rationality errors in the Standard Dispute. Some of them I've already covered, and some of them I've yet to cover; likewise the remedies.
But for now, I would just like to point out—in a mournful sort of way—that Albert and Barry seem to agree on virtually every question of what is actually going on inside the forest, and yet it doesn't seem to generate any feeling of agreement.
Arguing about definitions is a garden path; people wouldn't go down the path if they saw at the outset where it led. If you asked Albert (Barry) why he's still arguing, he'd probably say something like: "Barry (Albert) is trying to sneak in his own definition of 'sound', the scurvey scoundrel, to support his ridiculous point; and I'm here to defend the standard definition."
But suppose I went back in time to before the start of the argument:
(Eliezer appears from nowhere in a peculiar conveyance that looks just like the time machine from the original 'The Time Machine' movie.)
Barry: "Gosh! A time traveler!"
Eliezer: "I am a traveler from the future! Hear my words! I have traveled far into the past—around fifteen minutes—"
Albert: "Fifteen minutes?"
Eliezer: "—to bring you this message!"
(There is a pause of mixed confusion and expectancy.)
Eliezer: "Do you think that 'sound' should be defined to require both acoustic vibrations (pressure waves in air) and also auditory experiences (someone to listen to the sound), or should 'sound' be defined as meaning only acoustic vibrations, or only auditory experience?"
Barry: "You went back in time to ask us that?"
Eliezer: "My purposes are my own! Answer!"
Albert: "Well... I don't see why it would matter. You can pick any definition so long as you use it consistently."
Barry: "Flip a coin. Er, flip a coin twice."
Eliezer: "Personally I'd say that if the issue arises, both sides should switch to describing the event in unambiguous lower-level constituents, like acoustic vibrations or auditory experiences. Or each side could designate a new word, like 'alberzle' and 'bargulum', to use for what they respectively used to call 'sound'; and then both sides could use the new words consistently. That way neither side has to back down or lose face, but they can still communicate. And of course you should try to keep track, at all times, of some testable proposition that the argument is actually about. Does that sound right to you?"
Albert: "I guess..."
Barry: "Why are we talking about this?"
Eliezer: "To preserve your friendship against a contingency you will, now, never know. For the future has already changed!"
(Eliezer and the machine vanish in a puff of smoke.)
Barry: "Where were we again?"
Albert: "Oh, yeah: If a tree falls in the forest, and no one hears it, does it make a sound?"
Barry: "It makes an alberzle but not a bargulum. What's the next question?"
This remedy doesn't destroy every dispute over categorizations. But it destroys a substantial fraction.
I find it quite interesting that despite the two above posts having very strongly contradictory points they both a large number of upvotes are are both at 100% positive (15 and 8 at the time of writing). I wonder whether the community's opinion has shifted over the years, or whether lw voters just think both points are well put and are very reluctant to downvote things based on disagreeing with a point.
I think it's very much a case of well-put arguments.
I can certainly see how pragmatically, definitions can clearly matter. Heck, we have laws that are very picky about them, because we need a very specific set of rules so that crinimals/those falsely accused are clearly in one category or another, and to make the law above debate.
At the same time, asking whether something is against the law, whether it fits into the category of "murder" for example, is simly arguing whether it is case considered worthy, by those who wrote the law, of punishment.
Both arguments are very well explained by the two comments.