Good point. I think the main similarity derives from a specific understanding/definition of harm that holds that harming another is acting counter to another's preferences, in some sense. In that way then, it's similar to (the OP's trouble in getting his interlocutors to understand) preferences being sustained after one's death.