"our emotions arise from our models of reality. If I believe that my dead brother has been discovered alive, I will be happy"
Fallacy of the single cause. Knowledge of the physical fact of his being alive does not completely determine your response of being happy, many other things come into this, of which at least a few are non-rational. Maybe your brother is a convicted serial killer who recently escaped from detention, killed a few more people according to his old habits and is now reported to be alive only by virtue of having escaped a police hunt through the nearby forest (with officers ordered to shoot on sight). Yet still you may be happy to hear he's alive (and here comes the usual explanation people give, the real explanation) "because he's your brother". This is considered to be the most important factor in your being happy in this case - "because he's your brother" - and it encodes some non-rational baggage together with some arguably rational things (like an evolutionary preference to support the survival of your kin's genes etc.).
The fact is that any human preference results from multiple causes and at least one of those will always be non-rational (which is to say I don't know of even one single example where this was not the case) and will open said preference to being labelled "non-rational".
Reason is just a tool. Before you decide what to use the tool for, you have to have non-rational preferences about which things to even try to do. For example, first you have the non-rational desire to predict the future behaviour of physical systems with high accuracy and only afterwards do you employ rational methods to achieve that (which leads to science). The only rational part is what you're doing after you've established your fundamental goal. The fundamental goal itself can't be rational insofar as it can't be derived logically from any antecedents. Even if your desire to predict things was based on your desire to survive and even if the desire of the individual to survive could be justified on the basis of the evolutionary goal for the species to survive, you still end up at a point where you can no longer offer any justifications. Why should your species survive and not others? Maybe you think your species has the highest capacity of ensuring the survival of life-in-general in the universe for the longest time? But even then, why should life-in-general survive? Just because. Non-rationally. :)
Good catch; reason cannot determine our end goals. Eliezer covers that a just a few essays down the road.
If I believe that my dead brother has been discovered alive, I will be happy.
This was actually a personal statement, not a general hypothetical; his brother died three years before he wrote that essay, and wasn't a serial killer. But Eliezer would agree that death is just plain bad; that's a terminal value that doesn't have--or need--rational justification.
Since curiosity is an emotion, I suspect that some people will object to treating curiosity as a part of rationality. A popular belief about “rationality” is that rationality opposes all emotion—that all our sadness and all our joy are automatically anti-logical by virtue of being feelings. Yet strangely enough, I can’t find any theorem of probability theory which proves that I should appear ice-cold and expressionless.
When people think of “emotion” and “rationality” as opposed, I suspect that they are really thinking of System 1 and System 2—fast perceptual judgments versus slow deliberative judgments. System 2’s deliberative judgments aren’t always true, and System 1’s perceptual judgments aren’t always false; so it is very important to distinguish that dichotomy from “rationality.” Both systems can serve the goal of truth, or defeat it, depending on how they are used.
For my part, I label an emotion as “not rational” if it rests on mistaken beliefs, or rather, on mistake-producing epistemic conduct. “If the iron approaches your face, and you believe it is hot, and it is cool, the Way opposes your fear. If the iron approaches your face, and you believe it is cool, and it is hot, the Way opposes your calm.” Conversely, an emotion that is evoked by correct beliefs or truth-conducive thinking is a “rational emotion”; and this has the advantage of letting us regard calm as an emotional state, rather than a privileged default.
So is rationality orthogonal to feeling? No; our emotions arise from our models of reality. If I believe that my dead brother has been discovered alive, I will be happy; if I wake up and realize it was a dream, I will be sad. P. C. Hodgell said: “That which can be destroyed by the truth should be.” My dreaming self’s happiness was opposed by truth. My sadness on waking is rational; there is no truth which destroys it.
Rationality begins by asking how-the-world-is, but spreads virally to any other thought which depends on how we think the world is. Your beliefs about “how-the-world-is” can concern anything you think is out there in reality, anything that either does or does not exist, any member of the class “things that can make other things happen.” If you believe that there is a goblin in your closet that ties your shoes’ laces together, then this is a belief about how-the-world-is. Your shoes are real—you can pick them up. If there’s something out there that can reach out and tie your shoelaces together, it must be real too, part of the vast web of causes and effects we call the “universe.”
Feeling angry at the goblin who tied your shoelaces involves a state of mind that is not just about how-the-world-is. Suppose that, as a Buddhist or a lobotomy patient or just a very phlegmatic person, finding your shoelaces tied together didn’t make you angry. This wouldn’t affect what you expected to see in the world—you’d still expect to open up your closet and find your shoelaces tied together. Your anger or calm shouldn’t affect your best guess here, because what happens in your closet does not depend on your emotional state of mind; though it may take some effort to think that clearly.
But the angry feeling is tangled up with a state of mind that is about how-the-world-is; you become angry because you think the goblin tied your shoelaces. The criterion of rationality spreads virally, from the initial question of whether or not a goblin tied your shoelaces, to the resulting anger.
Becoming more rational—arriving at better estimates of how-the-world-is—can diminish feelings or intensify them. Sometimes we run away from strong feelings by denying the facts, by flinching away from the view of the world that gave rise to the powerful emotion. If so, then as you study the skills of rationality and train yourself not to deny facts, your feelings will become stronger.
In my early days I was never quite certain whether it was all right to feel things strongly—whether it was allowed, whether it was proper. I do not think this confusion arose only from my youthful misunderstanding of rationality. I have observed similar troubles in people who do not even aspire to be rationalists; when they are happy, they wonder if they are really allowed to be happy, and when they are sad, they are never quite sure whether to run away from the emotion or not. Since the days of Socrates at least, and probably long before, the way to appear cultured and sophisticated has been to never let anyone see you care strongly about anything. It’s embarrassing to feel—it’s just not done in polite society. You should see the strange looks I get when people realize how much I care about rationality. It’s not the unusual subject, I think, but that they’re not used to seeing sane adults who visibly care about anything.
But I know, now, that there’s nothing wrong with feeling strongly. Ever since I adopted the rule of “That which can be destroyed by the truth should be,” I’ve also come to realize “That which the truth nourishes should thrive.” When something good happens, I am happy, and there is no confusion in my mind about whether it is rational for me to be happy. When something terrible happens, I do not flee my sadness by searching for fake consolations and false silver linings. I visualize the past and future of humankind, the tens of billions of deaths over our history, the misery and fear, the search for answers, the trembling hands reaching upward out of so much blood, what we could become someday when we make the stars our cities, all that darkness and all that light—I know that I can never truly understand it, and I haven’t the words to say. Despite all my philosophy I am still embarrassed to confess strong emotions, and you’re probably uncomfortable hearing them. But I know, now, that it is rational to feel.