If they can't stop students from using Wikipedia, pretty soon schools will be reduced from teaching how to gather facts, to teaching how to think!

I think I hear you, but this comment is way confusing.

I missed where Vladimir made that suggestion, though I'm sure others have. You can have an irrational value, if it's really a means and not an end (which is another value), but you don't recognize that, and call the means a value itself. Means to an end can of course be evaluated as rational. If anyone made the suggestion you mention, they probably presumed a single "basic" value of preserving lives, and considered the method of deciding to be a means, but denoted as a value.

(Of course, a value can be both a means and an end, which presents fun new complications...)

I don't get you

There is nothing intrinsically irrational about any action, rationality or irrationality depends on preference, which is the point I was trying to communicate. Any question about "rationality" of a decision is a question about correctness of preference-optimization. So, my reply to your original question is that the question is ill-posed, and the content of the reply was explanation as to why.

Okay, so I'll ask again: why couldn't the humans real preference be to not sacrifice the children?
[...]
In the latter case, you still haven't explained why giving up the children is winning, and not doing so is not winning.

It seems like you are seeing my replies as soldier-arguments for the object-level question about the sacrifice of children, stumped on a particular conclusion that sacrificing children is right, while I'm merely giving opinion-neutral meta-comments about the semantics of such opinions. (I'm not sure I'm reading this right.)

You can't decide your preference, preference is not what you actually do, it is what you should do.

You haven't really elucidated this. You're either pulling an ought out of nowhere, or you're saying "preference is what you should do if you want to win".

Preference defines what constitutes winning, your actions rank high in the preference order if they determine the world high in preference order. Preference can't be reduced to winning or actions, as these all are the sides of the same structure.

Wonderful post.

Because the brain is a hodge podge of dirty hacks and disconnected units, smoothing over and reinterpreting their behaviors to be part of a consistent whole is necessary to have a unified 'self'. Drescher makes a somewhat related conjecture in "Good and Real", introducing the idea of consciousness as a 'Cartesian Camcorder', a mental module which records and plays back perceptions and outputs from other parts of the brain, in a continuous stream. It's the idea of "I am not the one who thinks my thoughts, I am the one who hears my thoughts", the source of which escapes me. Empirical support of this comes from the experiments of Benjamin Libet, which show that a subconscious electrical processes precede conscious actions - implying that consciousness doesn't engage until after an action has already been decided. If this is in fact how we handle internal information - smoothing out the rough edges to provide some appearance of coherence, it shouldn't be suprising that we tend to handle external information in the same matter.

Even this language, I suspect, is couched in a manner that expresses Cartesian Materialist remnants. One of the most interesting things about Dennett is that he believes in free will, despite his masterful grasp of the disunity of conscious experience and action. This, I think, is because he recognizes an important fact: we have to redefine the conscious self as something spaced out over time and location (in the brain), not as the thing that happens AFTER the preceding neuronal indicators.

But perhaps I'm misinterpreting your diction.

Preference of a given human is defined by their brain, and can be somewhat different from person to person, but not too much. There is nothing "objective" about this preference, but for each person there is one true preference that is their own, and same could be said for humanity as a whole, with the whole planet defining its preference, instead of just one brain. The focus on the brain isn't very accurate though, since environment plays its part as well.

I can't do justice to the centuries-old problem with a few words, but the idea is more or less this. Whatever the concept of "preference" means, when the human philosophers talk about it, their words are caused by something in the world: "preference" must be either a mechanism in their brain, a name of their confusion, or something else. It's not epiphenomenal. Searching for the "ought" in the world outside human minds is more or less a guaranteed failure, especially if the answer is expected to be found explicitly, as an exemplar of perfection rather than evidence about what perfection is, to be interpreted in nontrivial way. The history of failure to find an answer while looking in the wrong place doesn't prove that the answer is nowhere to be found, that there is now positive knowledge about the absence of the answer is the world.

jwdink, I don't think Vladimir Nesov is making an Is-Ought error. Think of this: You have values (preferences, desired ends, emotional "impulses" or whatever) which are a physical part of your nature. Everything you decide to do, you do because you Want to. If you refuse to acknowledge any criteria for behavior as valuable to you, you're saying that what feels valuable to you isn't valuable to you. This is a contradiction!

An Is-Ought problem arises when you attempt to derive a Then without an If. Here, the If is given: If you value what you value, then you should do what is right in accordance with your values.

This is a form of the general question "What's so bad about X?" with pain as X.

For any X, we can ask "What's so bad about X", receive an answer X2, and ask "What's so bad about X2", ad infinitum. The three most common responses are semantic stopsigns, moral nihilism, and admitting you need to ask the question more rigorously.

Once phrased more rigorously, the problem becomes easier, transforming into some combination of:

"Why do people dislike pain?", to which the answer is that it's hard-wired into the brain in some way a neurologist could probably explain, probably similar to how it's hard-wired to dislike things that taste bitter.

"Why do people call pain bad?", to which the answer is that most people think as emotivists, and call pain bad because they dislike it.

"Why is pain bad in Moral System Y?", to which the answer is that you'd have to ask the people in moral system Y, and they'll give you their moral system's answer. I think a lot of the better moral system would have it as an axiom. They probably make it an axiom because most moral systems are linked in some way or another to what people do or don't like, whether they admit it or not.

"Why is there a strong negative qualia for pain instead of it just feeling like a little voice at the back of your head saying 'that's painful'?", to which the answer will remain mysterious until we understand qualia, but no more mysterious than any other sensation.