The following was originally going to be a top-level post, but I never posted it because I couldn't complete the proof of my assertion.
In his recent book I Am a Strange Loop, Douglas Hofstadter writes:
A spectacular evolutionary gulf opened up at some point as human beings were gradually separating from other primates: their category systems became arbitrarily extensible. Into our mental lives there entered a dramatic quality of open-endedness, an essentially unlimited extensibility, as compared with a very palpable limitedness in other species. Concepts in the brains of humans acquired the property that they could get rolled together with other concepts into larger packets, any such larger packet could then become a new concept in its own right. In other words, concepts could nest inside each other hierarchically, and such nesting could go on to arbitrary degrees. This reminds me—and I do not think it is pure coincidence—of the huge difference, in video feedback, between an infinite corridor and a truncated one.
In other words, Hofstadter sees a phase transition, a discontinuity, a binary division between the mental processes of humans and other species. Yet curiously, when he discusses the moral consideration we ought to give to various species, he advocates a continuum approach based on something like "capacity for friendship", thereby privileging species with K-strategies and/or pack-hunting tendencies for no very good reason that I can see.
To me, the implication of Hofstadter's phase transition is obvious: beings with arbitrary category systems get moral consideration; those with bounded systems do not. By "moral consideration", incidentally, I don't mean some sort of Kantian treating-as-ends-not-means (oh wait, Kant is irrelevant); rather I mean that when you're making some nice utilitarian calculation, you must consider the feelings/opinions of all humans involved, but should not factor in the preferences of (say) a dog.
This is not to say that animal cruelty for the hell of it is a good idea (though I think it should be legal). Many of us anthropomorphize animals, especially pets, to a huge extent, and doing "evil" to animals could easily lead to actual evil. On the other hand, if you're deciding between torturing a human or a googol kittens, go for the kittens.
On the other hand, if you're deciding between torturing a human or a googol kittens, go for the kittens.
That's not what you're saying. You're saying, "Torture kittens, or don't; it's all the same."
For a long time, I wanted to ask something. I was just thinking about it again when I saw that Alicorn has a post on a similar topic. So I decided to go ahead.
The question is: what is the difference between morally neutral stimulus responces and agony? What features must an animal, machine, program, alien, human fetus, molecule, or anime character have before you will say that if their utility meter is low, it needs to be raised. For example, if you wanted to know if lobsters suffer when they're cooked alive, what exactly are you asking?
On reflection, I'm actually asking two questions: what is a morally significant agent (MSA; is there an established term for this?) whose goals you would want to further; and having determined that, under what conditions would you consider it to be suffering, so that you would?
I think that an MSA would not be defined by one feature. So try to list several features, possibly assigning relative weights to each.
IIRC, I read a study that tried to determine if fish suffer by injecting them with toxins and observing whether their reactions are planned or entirely instinctive. (They found that there's a bit of planning among bony fish, but none among the cartilaginous.) I don't know why they had to actually hurt the fish, especially in a way that didn't leave much room for planning, if all they wanted to know was if the fish can plan. But that was their definition. You might also name introspection, remembering the pain after it's over...
This is the ultimate subjective question, so the only wrong answer is one that is never given. Speak, or be wrong. I will downvote any post you don't make.
BTW, I think the most important defining feature of an MSA is ability to kick people's asses. Very humanizing.