This is a form of the general question "What's so bad about X?" with pain as X.

For any X, we can ask "What's so bad about X", receive an answer X2, and ask "What's so bad about X2", ad infinitum. The three most common responses are semantic stopsigns, moral nihilism, and admitting you need to ask the question more rigorously.

Once phrased more rigorously, the problem becomes easier, transforming into some combination of:

"Why do people dislike pain?", to which the answer is that it's hard-wired into the brain in some way a neurologist could probably explain, probably similar to how it's hard-wired to dislike things that taste bitter.

"Why do people call pain bad?", to which the answer is that most people think as emotivists, and call pain bad because they dislike it.

"Why is pain bad in Moral System Y?", to which the answer is that you'd have to ask the people in moral system Y, and they'll give you their moral system's answer. I think a lot of the better moral system would have it as an axiom. They probably make it an axiom because most moral systems are linked in some way or another to what people do or don't like, whether they admit it or not.

"Why is there a strong negative qualia for pain instead of it just feeling like a little voice at the back of your head saying 'that's painful'?", to which the answer will remain mysterious until we understand qualia, but no more mysterious than any other sensation.

How does one define "bad" without "pain" or "suffering"? Seems rather difficult. Or: The question doesn't seem so much difficult as it is (almost) tautological. It's like asking "What, if anything, is hot about atoms moving more quickly?"

I understand your frustration, since we don't seem to be saying much to support our claims here. We've discussed relevant issues of metaethics quite heavily on Less Wrong, but we should be willing to enter the debate again as new readers arrive and raise their points.

However, there's a lot of material that's already been said elsewhere, so I hope you'll pardon me for pointing you towards a few early posts of interest right now instead of trying to summarize it in one go.

Torture vs. Dust Specks kicked off the arguing; Eliezer began arguing for his own position in Circular Altruism and The "Intuitions" Behind "Utilitarianism". Searching LW for keywords like "specks" or "utilitarian" should bring up more recent posts as well, but these three sum up more or less what I'd say in response to your question.

(There's a whole metaethics sequence later on (see the whole list of Eliezer's posts from Overcoming Bias), but that's less germane to your immediate question.)

All human preferences, in their exact form, are hidden. The complexity of human value is too great to comprehend it all explicitly with a merely human mind.

The problem is a confusion. Human preference is something implemented in the very real human brain.

Yes they could care about either outcome. The question is whether they did, whether their true hidden preferences said that a given outcome is preferable.

The analogy in the next paragraph was meant to clarify. Do you see the analogy?

A person in this analogy is an equations together with an algorithm for approximately solving that equation. Decisions that the person makes are the approximate solutions, while preference is the exact solution hidden in the equation that the person can't solve exactly. The decision algorithm tries to make decisions as close to the exact solution as it can. The exact solution is what the person should do, while the output of the approximate algorithm is what the person actually does.

I should have said "decreases personal utility." When I say rationality, I mean rationality. Decreasing personal utility is the opposite of "winning".

So if said planet decided that its preference was to perish, rather than sacrifice children, would this be irrational?

You can't decide your preference, preference is not what you actually do, it is what you should do, and it's encoded in your decision-making capabilities in a nontrivial way, so that you aren't necessarily capable of seeing what it is.

Compare preference to a solution to an equation: you can see the equation, you can take it apart on the constituent terms, but its solution is nowhere to be found explicitly. Yet this solution is (say) uniquely defined by the equation, and approximate methods for solving the equation (analogized to the actual decisions) tend to give their results in the general ballpark of the exact solution.

Some people would say that dying honorably is better than living dishonorably. I'm not endorsing this view, I'm just trying to figure out why it's irrational, while the utilitarian sacrifice of children is more rational.

Utilitarian calculation is a more rational process of arriving at a decision, while for the output of this process (a decision) for a specific question you can argue that it's inferior to the output of some other process, such as free-running deliberation or random guessing. When you are comparing the decisions of sacrifice of children and war to the death, first isn't "intrinsically utilitarian", and the second isn't "intrinsically emotional".

Which of the decision is (actually) the better one depends on the preferences of one who decides, and preferences are not necessarily reflected well in actions and choices. It's instrumentally irrational for the agent to choose poorly according to its preferences. Systematic processes for decision-making allow agents to explicitly encode their preferences, and thus avoid some of the mistakes made with ad-hoc decision-making. Such systematic processes may be constructed in preference-independent fashion, and then given preferences as parameters.

Utilitarian calculation is a systematic process for computing a decision in situations that are expected to break intuitive decision-making. The output of a utilitarian calculation is expected to be better than an intuitive decision, but there are situations when utilitarian calculation goes wrong. For example, the extent to which you value things could be specified incorrectly, or a transformation that computes how much you value N things based on how much you value one thing may be wrong. In other cases, the problem could be reduced to a calculation incorrectly, losing important context.

However, whatever the right decision is, there normally should be a way to fix the parameters of utilitarian calculation so that it outputs the right decision. For example, if the right decision in the topic problem is actually war to the death, there should be a way to more formally understand the situation so that the utilitarian calculation outputs "war to the death" as the right decision.