Taboo both "morality" and "logical" and you may find that you and Eliezer have no disagreement.
LessWrongers routinely disagree on what is meant by "morality". If you think "morality" is ambiguous, then stipulate a meaning ('morality₁ is...') and carry on. If you think people's disagreement about the content of "morality" makes it gibberish, then denying that there are moral truths, or that those truths are "logical," will equally be gibberish. Eliezer's general practice is to reason carefully but informally with something in the neighborhood of our colloquial meanings of terms, when it's clear that we could stipulate a precise definition that adequately approximates what most people mean. Words like 'dog' and 'country' and 'number' and 'curry' and 'fairness' are fuzzy (if not outright ambiguous) in natural language, but we can construct more rigorous definitions that aren't completely semantically alien.
Surprisingly, we seem to be even less clear about what is meant by "logic". A logic, simply put, is a set of explicit rules for generating lines in a proof. And "logic," as a human practice, is the use and creation of such rules. But people informally speak of things as "logical" whenever they have a 'logicalish vibe,' i.e., whenever they involve especially rigorous abstract reasoning.
Eliezer's standard use of 'logical' takes the 'abstract' part of logicalish vibes and runs with them; he adopts the convention that sufficiently careful purely abstract reasoning (i.e., reasoning without reasoning about any particular spatiotemporal thing or pattern) is 'logical,' whereas reasoning about concrete things-in-the-world is 'physical.' Of course, in practice our reasoning is usually a mix of logical and physical; but Eliezer's convention gives us a heuristic for determining whether some x that we appeal to in reasoning is logical (i.e., abstract, nonspatial) or physical (i.e., concrete, spatially located). We can easily see that if the word 'fairness' denotes anything (i.e., it's not like 'unicorn' or 'square circle'), it must be denoting a logical/abstract sort of thingie, since fairness isn't somewhere. (Fairness, unlike houses and candy, does not decompose into quarks and electrons.)
By the same reasoning, it becomes clear that things like 'difficulty' and 'the average South African male' and 'the set of prime numbers' and 'the legal system of Switzerland' are not physical objects; there isn't any place where difficulty literally is, as though it were a Large Object hiding someplace just out of view. It's an abstraction (or, in EY's idiom, a 'logical' construct) our brains posit as a tool for thinking, in the same fundamental way that we posit numbers, sets, axioms, and possible worlds. The posits of literary theory are frequently 'logical' (i.e., abstract) in Eliezer's sense, when they have semantic candidates we can stipulate as having adequately precise characteristics. Eliezer's happy to be big-tent here, because he's doing domain-general epistemology and (meta)physics, not trying to lay out the precise distinctions between different fields in academia. And doing so highlights the important point that reasoning about what's moral is not categorically unlike reasoning about what's difficult, or what's a planet, or what's desirable, or what's common, or what's illegal; our natural-language lay-usage may underdetermine the answers to those questions, but there are much more rigorous formulations in the same semantic neighborhood that we can put to very good use.
So if we mostly just mean 'abstract' and 'concrete,' why talk about 'logical' and 'physical' at all? Well, I think EY is trying to constrain what sorts of abstract and concrete posits we take seriously. Various concepts of God, for instance, qualify as 'abstract' in the sense that they are not spatial; and psychic rays qualify as 'concrete' in the sense that they occur in specific places; but based on a variety of principles (e.g., 'abstract things have no causal power in their own right' and 'concrete things do not travel faster than light' or 'concrete things are not irreducibly "mental"'), he seeks to tersely rule out the less realistic spatial and non-spatial posits some people make, so that the epistemic grown-ups can have a more serious discussion amongst themselves.
A metaphysical thesis: These regimentations of moral terms do not commit us to implausible magical objects like Divine Commands or Irreducible ‘Oughtness’ Properties
If oughtness, nornmativity, isn't irteducible, it's either reducible or nonexistent. If it's nonexistent, how can you have morality at all? If it's reducible, where's the reduction?
What do I mean by "morality isn't logical"? I mean in the same sense that mathematics is logical but literary criticism isn't: the "reasoning" we use to think about morality doesn't resemble logical reasoning. All systems of logic, that I'm aware of, have a concept of proof and a method of verifying with high degree of certainty whether an argument constitutes a proof. As long as the logic is consistent (and we have good reason to think that many of them are), once we verify a proof we can accept its conclusion without worrying that there may be another proof that makes the opposite conclusion. With morality though, we have no such method, and people all the time make moral arguments that can be reversed or called into question by other moral arguments. (Edit: For an example of this, see these posts.)
Without being a system of logic, moral philosophical reasoning likely (or at least plausibly) doesn't have any of the nice properties that a well-constructed system of logic would have, for example, consistency, validity, soundness, or even the more basic property that considering arguments in a different order, or in a different mood, won't cause a person to accept an entirely different set of conclusions. For all we know, somebody trying to reason about a moral concept like "fairness" may just be taking a random walk as they move from one conclusion to another based on moral arguments they encounter or think up.
In a recent post, Eliezer said "morality is logic", by which he seems to mean... well, I'm still not exactly sure what, but one interpretation is that a person's cognition about morality can be described as an algorithm, and that algorithm can be studied using logical reasoning. (Which of course is true, but in that sense both math and literary criticism as well as every other subject of human study would be logic.) In any case, I don't think Eliezer is explicitly claiming that an algorithm-for-thinking-about-morality constitutes an algorithm-for-doing-logic, but I worry that the characterization of "morality is logic" may cause some connotations of "logic" to be inappropriately sneaked into "morality". For example Eliezer seems to (at least at one point) assume that considering moral arguments in a different order won't cause a human to accept an entirely different set of conclusions, and maybe this is why. To fight this potential sneaking of connotations, I suggest that when you see the phrase "morality is logic", remind yourself that morality isn't logical.