I don't see how that follows.

My position is more-or-less the one argued by Marius in this thread. Especially the second posibility in this comment.

Also, I think it would be better to take all further discussion into that thread.

All I'm saying is that there needs to be an intelligence, some value-having agent or entity, in order for actions to be judged. If there is no intelligence, there are no values.

Judging requires an agent. But values does not. That just requires an object capable of representing information. The universe could have values built into it even without having intelligence to be judging with it. (Completely irrelevant observation.)

I could sit here and explain to you how the chess analogy fails, how the rationality analogy fails, et cetera and so on.

I would find it easier to believe you could if you had.

I avoid doing unto others what I would not wish done unto me because that policy, when shared by social animals like humans, leads to results I prefer. I have evolved that preference. I voluntarily cooperate because it is in my direct interest, i.e. fulfills my values and preferences.

Reasoning about preferences is still reasoning.

Are you asserting that being "moral" is just like a game with a set of agreed upon rules?

Agreed upon? Most of the game is in making up rules and forcing them on others despite their disagreement!

I'm arguing that there is a sense in which one "should" follow rules which has nothing to do with human-like agents laying down the law, thereby refuting NMJ's attempt at a link between objective morality and theism.

There are constraints on the rules games can have (eg fairness, a clear winner after finite time). There are constraints on rationality (eg avoidance of quodlibet). Likewise, there are constraints on the rules of moral reasoning. (eg. people cannot just make up their own morality and do what they want). Note that I am talking about metaethics here.

The rules of chess would certainly exist, as much as any other mathematical object does. Of course, not every mind would care to follow them...

I'll take the bait: A place where the idea of chess had never been thought of couldn't, by definition, contain a chess set. It could contain an artifact physically identical to one and which we could call one if we weren't being precise, but it would have to have come from some origin besides humans intending to build a chess set. Thus would be perfectly reasonable for the two to speculate (and be right or wrong) about what (if any) use it was meant for by its actual builders (if any).

To expound on JoshuaZ's point, would chess have rules even if there were no minds?

Are the rules of chess objective and independent of anyone actually, y'know, knowing them?

Nothing I have to say about morality or metaethics hinges on that one way or the other.

'Best' here means most advantageous to the agents involved - both the focal agent, and the other agents who join in the convention. What I have in mind here is something like the Nash Bargaining Solution, which is both unique and (in some sense) optimal.

"Morally permissible" by the best code, which is also the code mentioned in the first paragraph. What I am claiming is that the 'moral should' can be reduced to the 'practical self-interested should' if (and it is a big if) it is reasonable to treat the other agents as rational and well-informed.

Actually, I am saying something about ultimate ends, at least indirectly. My position only makes sense if long-term ultimate ends become somehow 'spookily metaphysically privileged' over short-term ultimate ends.

My position still contains 'spookiness' but it is perhaps a less arbitrary kind of 'magic' - I'm talking about time-scales rather than laws inscribed on tablets.

That one agent's preferences differ greatly from the norm does not automatically make cooperation impossible. In a non-zero-sum game of perfect information, there is always a gain to be made by cooperating. Furthermore, it is usually possible to restructure the game so that it is no longer zero-sum.

For example, a society confronting a would-be suicide bomber will (morally and practically) incarcerate him, if it has the information and the power to do so. And, once thwarted from his primary goal, the would-be bomber may find that he now has some common interests with his captors. The game is no longer zero-sum.

So I don't think that divergent interests are a fatal objection to my scheme. What may be fatal is that real-world games are not typically games with perfect information. Sometimes, in the real world, it is advantageous to lie about your capabilities, values, and intentions. At least advantageous in the short term. Maybe not in the long term.

I don't think that sentence is true, as there are many people (e.g. suicide bombers) whose evaluations of their long-term interest are significant outliers from other agents.

That's right, but this exception (people whose interests are served by violating the moral norm) itself has a large exception, which is that throughout most of the suicide bomber's life, he (rightly) respects the moral norm. Bad people can't be bad every second of their lives - they have to behave themselves the vast majority of the time if for no other reason than to survive until the next opportunity to be bad. The suicide bomber has no interest in surviving once he presses the button, but for every second of his life prior to that, he has an interest in surviving.

And the would-be eventual suicide bomber also, through most of his life, has no choice but to enforce moral behavior in others if he wants to make it to his self-chosen appointment with death.

If we try to imagine someone who never respects recognizable norms - well, it's hard to imagine, but for one thing they would probably make most "criminally insane" look perfectly normal and safe to be around by contrast.

Interestingly, it seems to me that this definition would produce meta-maxims more readily than normal ones - "find out what your local society's norms are and stick to those" rather than "don't eat babies", for example.

Yes, it does produce meta-maxims in some sense. But meta-maxims can provide useful guidance. "Do unto others ..." is a meta-maxim. So is "Don't steal (anything from anybody)" or "Don't lie (about anything to anyone)"

As to whether this definition leads to the meta-maxim "find out what your local society's norms are and stick to those", that is explicitly not a consequence of this definition. Instead, the meta-maxim would be "work out what your local society's norms should be and set an example by doing that".

I didn't downvote, but you didn't address the crypto-Kantian angle at all. So it seems like you stopped reading too soon.

Yes, in a society with both human and non-human agents, if the humans contribute nothing at all to the non-humans, and only consume resources that might have been otherwise used, then the non-humans will judge the humans to be worthless. Worse than worthless. A drain to be eliminated.

But there is nothing special about my version of ethics in this regard. It is a problem that must be faced in any system of ethics. It is the FAI problem. Eliezer's solution is apparently to tell the non-human agents as they are created "Humans are to be valued. And don't you forget it when you self-modify." I think that a better approach is to make sure that the humans actually contribute something tangible to the well-being of the non-humans.

Perhaps neither approach is totally safe in the long run.

What does it mean in a concrete and substantive sense for pi to be an irrational number?

I can understand what physical conditions you are describing when you say "two rocks". What does it mean, in a concrete and substantive sense, for murder to be "wrong"?

I can give you two answers to this, one which maps better to this community and one which fits better with the virtue ethics tradition.

1. There exists (in the sense that mathematical functions exist) a utility function labeled 'morality' in which actions labeled 'murder' bring the universe into a state of lower-utility. I make no particular claims about the proper way to choose such a utility function, just that there is one that is properly called 'morality', and moral disputes can be characterized as either disputes over which function to call 'morality' or disputes over what the output of that function would be given certain inputs.

2. 'Good' and 'bad' are always evaluated in terms of effects upon a particular thing; a good hammer is one which optimally pounds in nails, a good horse is fast and strong, and a good human experiences eudaimonia. Murder is the sort of thing that makes one a bad human; it makes one less virtuous and thus less able to experience eudaimonia.

It could be the case that the terms 'good', 'bad', and 'eudaimonia' should be evaluated based on the preferences of an agent. But in that case it that does not make it any less the case that moral facts are facts about the world that one could be wrong about. For instance, if I prefer to live, I should not drink drain cleaner. If I thought it was good to drink drain cleaner, I would be wrong according to my own preferences, and an outside agent with different preferences could tell me I was objectively wrong about what's right for me to do.

As a side note, 'murder' is normative; it is tautologically wrong. Denying wrongness in general denies the existence of murder. It might be better to ask, "What does it mean for a particular sort of killing to be 'wrong'?", or else "What does it mean for a killing to be murder?"

So that we can know what we will do.

I assume they would be fine, being immortal.