Well, I admit to being one of the approximately seven billion humans who can't prove their utility functions from first principles. But I think there's a very convincing argument that consciousness is in fact what we're actually looking for and naturally taking into account.
Happiness only is happiness, and pain only is pain, insofar as it is perceived by awareness. If a scientist took a nerve cell with a pain receptor, put it in a Petri dish, and stimulated it for a while, I wouldn't consider this a morally evil act.
I find in my own life that different levels of awareness correspond to different levels of suffering. Although something bad happening to me in a dream is bad, I don't worry about it nearly as much as I would if it happened when I was awake and fully aware. Likewise, if I'm zonked out on sedatives, I tend to pay less attention to my own pain.
I hypothesize that different animals have different levels of awareness, based on intuition and my knowledge of their nervous systems. In this case, they would be able to experience different levels of suffering. What I meant by saying my utility function multiplied suffering by awareness would have been better phrased as:
Suffering = bad things*awareness
while trying to minimize suffering. This is why, for example, doing all sorts of horrible things to a rock is a morally neutral act, doing them to an insect is probably bad but not anything to lose sleep over, and doing them to a human is a moral problem even if it's a human I don't personally like.
Your paperclip example is a classical problem called the utility monster. I don't really have any especially brilliant solution beyond what has already been said about the issue. To some degree I bite the bullet: if there was some entity whose nervous system was so acute that causing it the slightest amount of pain would correspond to 3^^^3 years of torture for a human being, I'd place high priority on keeping that entity happy.
Well, I admit to being one of the approximately seven billion humans who can't prove their utility functions from first principles.
But you seem to think (and correct me if I'm misinterpreting) that it would be better if we could. I'm not so sure. And further you seem to think that given that we can't, it's still better to override our felt/intrinsic preferences that are hard to fully justify with unnatural preferences that have the sole advantage of being easier to express in simple sentences.
Now I'm not sure you're actually claiming this but with the ...
I have noticed that among philosophers, vegetarianism of one form or another is quite common. In fact, I became a vegetarian (technically a pescetarian) myself partly out of respect for an undergraduate philosophy professor. I am interested in finding out if there is a similar disproportion in the Less Wrong community.
I didn't request that this go into Yvain's survey because I want more information than just what animal products you do or don't eat; I'd also like to see nuances of the reasons behind your diet. There are a lot more shades than carnivore/vegetarian/vegan - if you want to be a vegetarian but are allergic to soy and gluten, that's a compelling reason to diversify protein sources, for instance. I'd also like to hear about if you avoid any plant foods (if you think they're farmed in a way that's environmentally destructive or that hurts people or if you have warm fuzzy feelings for plants, maybe). Here are some questions that come to mind: