There's an argument in the metaethics sequence, to the effect that there are no universally compelling moral arguments. This argument seems to be an important cashed thought (in don't mean that in any pejorative sense) in LW discussions of morality. This argument also seems to me to be faulty. Can anyone help me see what I'm missing?
The argument is from No Universally Compelling Arguments:
Yesterday, I proposed that you should resist the temptation to generalize over all of mind design space. If we restrict ourselves to minds specifiable in a trillion bits or less, then each universal generalization "All minds m: X(m)" has two to the trillionth chances to be false, while each existential generalization "Exists mind m: X(m)" has two to the trillionth chances to be true.
This would seem to argue that for every argument A, howsoever convincing it may seem to us, there exists at least one possible mind that doesn't buy it.
The central inference in the argument seems to me to go like this:
P1) Any universal generalization over minds ('All minds m: X(m)') is very unlikely to be true.
P2) A purportedly universally compelling moral argument has the form 'All minds m: X(m)'
C) A purportedly universally compelling moral argument is very unlikely to be true.
The reason I think this is faulty is that P1 is itself an argument of the form 'All minds m: X(m)', that is, it's a universal generalization over minds. If that's so, then P1 is very unlikely to be true, and we shouldn't accept the argument. In order to save the argument, we would have to weaken P1 to cover a more specific set of generalizations over minds (so that P1 itself is excluded) but if we do this, then the argument is invalid, since universally compelling moral arguments may end up excluded as well. We might have good reasons for thinking they won't be, but no such reasons are given in the sequence post.
Oh my, the confusion.
First off, the quoted argument was, as far as I can tell, entirely meant as an illustrative abstraction. The culprit here is the devious function X().
Suppose I take the set of all possible logically coherent statements that could be made about any given mind. Within this set, 'X' is any given statement about one mind. X(m) represents whether this given statement is True, False or Undefined / Undecidable for this mind 'm'.
For all X1..Xn, for a given mind 'm1', find all the X that are true. Then for all X() for m2, find those that are tr...
If it's worth saying, but not worth its own post (even in Discussion), then it goes here.