I find Eliezer's explanation of what "should" means to be unsatisfactory, and here's an attempt to do better. Consider the following usages of the word:
- You should stop building piles of X pebbles because X = Y*Z.
- We should kill that police informer and dump his body in the river.
- You should one-box in Newcomb's problem.
All of these seem to be sensible sentences, depending on the speaker and intended audience. #1, for example, seems a reasonable translation of what a pebblesorter would say after discovering that X = Y*Z. Some might argue for "pebblesorter::should" instead of plain "should", but it's hard to deny that we need "should" in some form to fill the blank there for a translation, and I think few people besides Eliezer would object to plain "should".
Normativity, or the idea that there's something in common about how "should" and similar words are used in different contexts, is an active area in academic philosophy. I won't try to survey the current theories, but my current thinking is that "should" usually means "better according to some shared, motivating standard or procedure of evaluation", but occasionally it can also be used to instill such a standard or procedure of evaluation in someone (such as a child) who is open to being instilled by the speaker/writer.
It seems to me that different people (including different humans) can have different motivating standards and procedures of evaluation, and apparent disagreements about "should' sentences can arise from having different standards/procedures or from disagreement about whether something is better according to a shared standard/procedure. In most areas my personal procedure of evaluation is something that might be called "doing philosophy" but many people apparently do not share this. For example a religious extremist may have been taught by their parents, teachers, or peers to follow some rigid moral code given in their holy books, and not be open to any philosophical arguments that I can offer.
Of course this isn't a fully satisfactory theory of normativity since I don't know what "philosophy" really is (and I'm not even sure it really is a thing). But it does help explain how "should" in morality might relate to "should" in other areas such as decision theory, does not require assuming that all humans ultimately share the same morality, and avoids the need for linguistic contortions such as "pebblesorter::should".
There are two things (at least) you could mean here, one of which I agree with. You could mean "If you say 'You should vote your pocket book' yet you can't clearly explain what you mean by 'you should vote your pocket book', then you can have no confidence that you mean anything particularly coherent by 'you should vote your pocket book'." I agree with this, but it seems to be beside the point.
But the issue of tabooing is different. If you mean "If you say 'you should vote your pocket book', but cannot taboo or define the word 'should' then you can have no confidence that you mean anything particularly coherent by 'you should vote your pocket book'." If this is what you mean, then I disagree and I think this is strongly contra-indicated by the linguistic practices of everyone around us. How many people, after all, could come up with a taboo or definition for 'should'? Yet I imagine this philosophical oversight would not prevent anyone (if they were so capable) from explaining what 'you should vote your pocket book' means.
I agree that if I am confident that I know what the sentence means (as you seem to be), that should increase my confidence that I also know what "should" means in that sentence (ditto "vote" and "pocketbook").
But I'm not confident that I know what that sentence means, propositionally anyway, and your stated reasons for such confidence (that lots of people use the sentence without noticing a problem) don't seem compelling to me, because lots of people regularly utter all kinds of sentences whose propositional content is deeply... (read more)