Why and how does anyone ever "do something they know they shouldn't", or "want something they know is wrong"? Does the notion of morality-as-preference really add up to moral normality?
It's all about delicious versus nutritious. That is, these conflicts are conflicts between different time horizons, or different discount values for future costs and benefits. Evolution has shaped our time horizon for making relatively short term decisions (Eat the pie now. It will taste good. There may not be another chance.), but we live in a world where a longer term is more appropriate (The pie may taste good, but it isn't good for my health. Also, I may benefit in the long term by giving the pie to somebody else.).
Followup to: The Bedrock of Fairness
Discussions of morality seem to me to often end up turning around two different intuitions, which I might label morality-as-preference and morality-as-given. The former crowd tends to equate morality with what people want; the latter to regard morality as something you can't change by changing people.
As for me, I have my own notions, which I am working up to presenting. But above all, I try to avoid avoiding difficult questions. Here are what I see as (some of) the difficult questions for the two intuitions:
Part of The Metaethics Sequence
Next post: "Is Morality Preference?"
Previous post: "The Bedrock of Fairness"