before we taboo a word, we need to make sure we fully understand what we mean by it.
Well, yes. As you say, tabooing a word with some more detailed description of something I didn't actually mean in the first place is an error... which is to say, I shouldn't do it... which is to say, it has negative expected value. (At least, that's what I think I meant. Perhaps I'm mistaken.)
But if I don't know what I mean by a word, and therefore can't correctly taboo it, continuing to use the word unreflectively doesn't really help us communicate clearly either. (Is that to say I shouldn't do it? Maybe.)
So, sure, maybe the various uses of "should" have some core commonality and sufficient analysis of that problem will make explicit some important insight about the nature of whatever that core commonality refers to which is currently implicit in our language use. In which case continued analysis of that core commonality might reveal useful insights and is therefore worth doing.
But that still doesn't seem like a reason to use "should" in my conversations once I've convinced myself that I don't know what I mean by it.
So, what happens if I do something else instead?
Well, for example, I was about to write "So, what should I do instead?" and, recognizing the irony, stopped and rethought what question I wanted to ask. Did I in the process taboo "should"? Perhaps not... it's quite possible that the question I asked is importantly different from the one I initially meant to ask. (For example, the question I asked is explicitly consequentialist, and the one I initially meant to ask is not, which might be a change of meaning or it might not be.)
Am I worse off for having done this? Would I have been better off to retain the original wording?
Well, if it turns out that there is value to a deontological view of ethics, then my replacing my original vague statement with an explicitly consequentialist statement has negative expected value. If not, then it has positive value. Either I think that's more likely than the alternative, or less likely. But it seems like I pretty much have to make a choice here based on incomplete information and my best guess.
To say "I don't know enough to taboo 'should' so I'll keep using it the way I'm accustomed to" seems unjustified.
But if I don't know what I mean by a word, and therefore can't correctly taboo it, continuing to use the word unreflectively doesn't really help us communicate clearly either.
I don't think that's quite right: there's enough of a gap between 'knowing how to use a word' and 'knowing how to define a word or replace it with other language' that I don't think it's reasonable to take the latter as decisive for whether or not we should feel comfortable carrying on with a discussion.
For example, if you asked me to taboo 'should' I would be flabbergasted. I woul...
I find Eliezer's explanation of what "should" means to be unsatisfactory, and here's an attempt to do better. Consider the following usages of the word:
All of these seem to be sensible sentences, depending on the speaker and intended audience. #1, for example, seems a reasonable translation of what a pebblesorter would say after discovering that X = Y*Z. Some might argue for "pebblesorter::should" instead of plain "should", but it's hard to deny that we need "should" in some form to fill the blank there for a translation, and I think few people besides Eliezer would object to plain "should".
Normativity, or the idea that there's something in common about how "should" and similar words are used in different contexts, is an active area in academic philosophy. I won't try to survey the current theories, but my current thinking is that "should" usually means "better according to some shared, motivating standard or procedure of evaluation", but occasionally it can also be used to instill such a standard or procedure of evaluation in someone (such as a child) who is open to being instilled by the speaker/writer.
It seems to me that different people (including different humans) can have different motivating standards and procedures of evaluation, and apparent disagreements about "should' sentences can arise from having different standards/procedures or from disagreement about whether something is better according to a shared standard/procedure. In most areas my personal procedure of evaluation is something that might be called "doing philosophy" but many people apparently do not share this. For example a religious extremist may have been taught by their parents, teachers, or peers to follow some rigid moral code given in their holy books, and not be open to any philosophical arguments that I can offer.
Of course this isn't a fully satisfactory theory of normativity since I don't know what "philosophy" really is (and I'm not even sure it really is a thing). But it does help explain how "should" in morality might relate to "should" in other areas such as decision theory, does not require assuming that all humans ultimately share the same morality, and avoids the need for linguistic contortions such as "pebblesorter::should".