I don't see anything here that is not a mixture of physical facts and logical facts (that is, truths about causal events and truths about premise-conclusion links). Physical computers within our universe may be neatly described by compact axioms. Logic (in my not-uncommon view) deals with semantic implication: what is true in a model given that the axioms are true of it. If you prove P!=NP using axioms that happen to apply to the computers of this universe then P!=NP for them as well, and the axioms will have been picked out to be applicable to real physics - a mixture of physical fact and logical fact. I don't know where logical facts are stored or what they are, just as I don't yet know what makes the universe real, although I repose some confidence that the previous two questions are wrong - but so far I'm standing by my view that truths are about causal events, logical implications, or some mix of the two.
Axioms are that which mathematicians use to talk about integers instead of something else. You could also take the perspective of trying to talk about groups of two pebbles as they exist in the real world, and wanting your axioms to correspond to their behavior. But when you stop looking at the real world and close your eyes and try to do math, then in order to do math about something, like about the integers, about these abstract objects of thought that you abstracted away from the groups of pebbles, you need axioms that identify the integers in mathspace. And having thus gained a subject of discourse, you can use the axioms to prove theorems that are about integers because the theorems hold wherever the axioms hold. And if those axioms are true of physical reality from the appropriate standpoint, your conclusions will also hold of groups of pebbles.
Perhaps more to the point, do you agree that there is a coherent meta-ethical position that does deserve to be called moral realism, which asserts that moral and meta-moral computations are about something outside of individual humans or humanity as a whole (even if we're not sure how that works)?
That depends; is morality a subject matter that we need premises to identify in subjectspace, in order to talk about morality rather than something else, stored in that same mysterious place as 2 + 2 = 4 being true of the integers but needing axioms to talk about the integers in the first place? Or are we talking about transcendent ineffable compelling stuff? The first view is, I think, coherent; I should think so, it's my own. The second view is not.
I don't see anything here that is not a mixture of physical facts and logical facts (that is, truths about causal events and truths about premise-conclusion links).
Eliezer, a couple of comments ago I switched my focus from whether there is more than just physical and logical facts to whether "morality" refers to something independent of humanity, like (as I claimed) "rationality", "integer" and "P!=NP" do. Sorry if I didn't make that clear, and I hope I'm not being logically rude here, but the topic is confusing t...
On Wei_Dai's complexity of values post, Toby Ord writes:
The kind of moral realist positions that apply Occam's razor to moral beliefs are a lot more extreme than most philosophers in the cited survey would sign up to, methinks. One such position that I used to have some degree of belief in is:
Strong Moral Realism: All (or perhaps just almost all) beings, human, alien or AI, when given sufficient computing power and the ability to learn science and get an accurate map-territory morphism, will agree on what physical state the universe ought to be transformed into, and therefore they will assist you in transforming it into this state.
But most modern philosophers who call themselves "realists" don't mean anything nearly this strong. They mean that that there are moral "facts", for varying definitions of "fact" that typically fade away into meaninglessness on closer examination, and actually make the same empirical predictions as antirealism.
Suppose you take up Eliezer's "realist" position. Arrangements of spacetime, matter and energy can be "good" in the sense that Eliezer has a "long-list" style definition of goodness up his sleeve, one that decides even contested object-level moral questions like whether abortion should be allowed or not, and then tests any arrangement of spacetime, matter and energy and notes to what extent it fits the criteria in Eliezer's long list, and then decrees goodness or not (possibly with a scalar rather than binary value).
This kind of "moral realism" behaves, to all extents and purposes, like antirealism.
I might compare the situation to Eliezer's blegg post: it may be that moral philosophers have a mental category for "fact" that seems to be allowed to have a value even once all of the empirically grounded surrounding concepts have been fixed. These might be concepts such as "would aliens also think this thing?", "Can it be discovered by an independent agent who hasn't communicated with you?", "Do we apply Occam's razor?", etc.
Moral beliefs might work better when they have a Grand Badge Of Authority attached to them. Once all the empirically falsifiable candidates for the Grand Badge Of Authority have been falsified, the only one left is the ungrounded category marker itself, and some people like to stick this on their object level morals and call themselves "realists".
Personally, I prefer to call a spade a spade, but I don't want to get into an argument about the value of an ungrounded category marker. Suffice it to say that for any practical matter, the only parts of the map we should argue about are parts that map-onto a part of the territory.