Didn't you just agree that the algorithm for sorting things into "right" and "not right" is different in different in different people?
Yes, and I also argued, repeatedly, against saying that such an algorithm constitutes either a definition or a meaning.
What if you are really wrong? What if you are the guy who is rounding the slave owners "property" and dutifully returning them to him?
Then I'm wrong about some fact that I used in translating my morality into actions, e.g. skin color determines intelligence.
Not necessarily. You could be wrong about morality itself. You could think property rights are more important than liberty, or that people are means not ends.
. So I really do have to stick with morality as the algorithm itself and not some run of it if I want consistency (though that's not strictly necessary).
Those are not your only choices.
You could be wrong about morality itself.
What sort of impact would being right or wrong about morality have that I could notice? For example, let's say someone thinks taxation is inherently morally wrong. What sort of observations are ruled out by this belief, such that making those observations would falsify the belief?
Derek Parfit has published his second book, "On What Matters". Here are reviews by Tyler Cowen and Peter Singer.