The way that we can resolve values vs. errors is by endorsing symmetries.

For example, Rawl's "veil of ignorance" enjoins us to design a society, on the assumption that we might be anyone in that society - we might have any degree of talent or disability, any taste or preferences, and so on. This is permutation symmetry.

If we have two situations that we believe are exactly analogous (for example, the trolley car problem and a similar problem with a subway car), then we call any systematic difference in our human intuitions an error, and we choose one of the two intuitions to endorse as applying to both cases. (I don't know that people systematically differ in their survey responses to questions about trolley car problems vs. subway car problems, but I wouldn't be surprised.)

In forming a notion of values and errors, we are choosing a priority order among various symmetries and various human intuitions. Utilitarians prioritize the analogy between flipping the switch and pushing the fat man over the intuition that we should not push the fat man.