This post has too many comments to show them all at once! Newcomers, please proceed in an orderly fashion to the newest welcome thread.
If you haven't, you should read Yvain's Consequentialism FAQ, which addresses some of these points in a little more detail.
Would you condone someone for forcing someone else to try chocolate, if that person believed it tasted bad, but loved it as soon as they tried it? If someone mentally deranged set themselves on fire and asked you not to save them, would you? If someone is refusing cancer treatment because "Science is evil", I at least would force the treatment on them. Would you force transhumanity on everyone who refused it, is probably a better question for LessWrong.
Preference utilitarianism works well for any situation you'll encounter in real life, but it's possible to propose questions it doesn't answer very well. A popular answer to the above question on LessWrong comes from the idea of coherent extrapolated volition (The paper itself may be outdated). Essentially, it asks what we would want were we better informed, more self-aware, and more the people we wished we were. In philosophy, this is called idealized preference theory. CEV probably says we shouldn't force someone to eat chocolate, because their preference for autonomy outweighs their extrapolated preference for chocolate. It probably says we should save the person on fire, since their non-mentally-ill extrapolated volition would want them to live, and ditto the cancer patient.
Forcing transhumanity on people is a harder question, because I'm not sure that everyone's preferences would converge in this case. In any event, I would not personally do it, because I don't trust my own reasoning enough.
But can you deny that there are people you think should have theirs changed?
I think all people, to the extent that they can be said to have utility functions, are wrong about what they want at least sometimes. I don't think we should change their utility function so much as implement their ideal preferences, not their stated ones.
I'm inclined to think that a large part of this community is.
is what? Is willing to change people's utility functions?
Forcing transhumanity on people
What does this even mean? Forcing immortality on people is at least a coherent notion, although I'm pretty sure most users around here support an individual's right to self-terminate. But if that was what was meant, calling it 'transhumanism' is a little off.
On the other hand, is this referring to something handled by the fun theoretic concept of a eudaimonic rate of intelligence increase?
Yes, I know, "jargon jargon jargon buzzword buzzword rationality," but I couldn't think of a better way to phrase that. Sorry.