Where I differ with you is in the number of black boxes. "We" don't have "a" black box. "Each" of us has our own black box.
This doesn't seem to be a point on which we differ at all. In this later comment I'm saying pretty much the same thing.
Indeed, I wouldn't be surprised if each of us has hundreds of processes that feel like they're calculating "morality", and aren't evaluating according to the same inputs. Some might have outputs that are not quite easy to directly compare, or impossible to.
OK. I see your other comment. I think I was mainly responding to this:
However, if one is to ask a moral question without including a specific group-referent (though usually, "all humans" or "most humans" is implicit) from which one can extract that objective algorithm that makes things moral or not
You can't extract "an" objective algorithm even if you do specify a group of people, unless your algorithm returns the population distribution of their moral evaluations, and not a singular moral evaluation. Any singular statistic would be one of an infinite set of statistics on that distribution.
I think there’s a confusion in our discussions of deontology and consequentialism. I’m writing this post to try to clear up that confusion. First let me say that this post is not about any territorial facts. The issue here is how we use the philosophical terms of art ‘consequentialism’ and ‘deontology’.
The confusion is often stated thusly: “deontological theories are full of injunctions like ‘do not kill’, but they generally provide no (or no interesting) explanations for these injunctions.” There is of course an equivalently confused, though much less common, complaint about consequentialism.
This is confused because the term ‘deontology’ in philosophical jargon picks out a normative ethical theory, while the question ‘how do we know that it is wrong to kill?’ is not a normative but a meta-ethical question. Similarly, consequentialism contains in itself no explanation for why pleasure or utility are morally good, or why consequences should matter to morality at all. Nor does consequentialism/deontology make any claims about how we know moral facts (if there are any). That is also a meta-ethical question.
Some consequentialists and deontologists are also moral realists. Some are not. Some believe in divine commands, some are hedonists. Consequentialists and deontologists in practice always also subscribe to some meta-ethical theory which purports to explain the value of consequences or the source of injunctions. But consequentialism and deontology as such do not. In order to avoid strawmaning either the consequentialist or the deontologist, it’s important to either discuss the comprehensive views of particular ethicists, or to carefully leave aside meta-ethical issues.
This Stanford Encyclopedia of Philosophy article provides a helpful overview of the issues in the consequentialist-deontologist debate, and is careful to distinguish between ethical and meta-ethical concerns.
SEP article on Deontology