Epistemic status: Many citations missing.

Itzhak Gilboa has defined an irrational decision as one where the decision-maker feels embarrassed when confronted with an analysis of their decision. Normativity is then advice to be less irrational (or, the practice of following all such advice in advance).

Is consequentialism normative? Are virtue-ethicists irrational?

I don't think that is inherently true. Batman can really, truly value not directly killing others, even if that ultimately leads to more death down the line. He simply has to center his values on himself - literally, to be self-centered - and specifically his own actions. It's conceivable to formulate a first-person utility function for Batman that takes in his experiences over the course of his life and assigns an arbitrarily low value to those that include actions where he kills someone.

Formal version

I claim that there exist utility functions for AIXI-like agents that depend on the action symbols of their interaction histories. In fact, a recent paper submission of mine (with Marcus Hutter) generalizes the AIXI model in a way that allows this sort of thing. It's not hard to do; it just leaves more "free parameters" than the reward-based utility function of the original AIXI, which makes it a much less specific theory, with many particular implementations that are not interesting.

Batman is not very consequentialist, even though he can sort-of be framed as a consequentialist from a first-person view. This first-person and action-oriented type of consequentialism is, however, incredibly permissive - whatever a person does, one might argue that they preferred it for its own sake[1]

For a subtly stronger type of consequentialism, we can be a little more strict, and allow Batman to care not about the internal choice to act but only the "realized" action itself and its consequences. For instance, perhaps Batman should not care whether he internally (almost metaphysically) chose to kill, but only whether he actually observed his hands strangling someone. But by designating "Batman's hands" as a particularly central focus of value in the universe, we still permit a lot of things that look unreasonable from the third person.

In other words, apparently-normative advice (e.g. "please just finish this by killing the Joker so that he doesn't break out of prison for the hundredth time and murder more innocents") may not be normative for a sufficiently stubborn egoist or solipsist.

However, I have an intuition that most of us aren't really that self-centered, if we thought it through. Really, we value the things that happen "out there" in the universe. 

One reason for this is that valuing acts (and all of virtue ethics) arises naturally as a heuristic for having good consequences on the universe. Therefore, I expect that nudging someone with "isn't the reason you decided killing was wrong really because you don't like for people to die?" may actually work, for genetic[2] reasons. Virtue ethics looks suspiciously like a confusion between instrumental and intrinsic value - it doesn't logically have to be, be one suspects it is in any given case.

This points at a deeper reason. Perhaps those beings that are selected for by optimization processes like evolution tend to maximize their influence. If so, the strictest form of consequentialism (actually caring about what happens out in the world, without being overly concerned about one's own actions) seems like a natural value system to arrive at - though perhaps, not necessarily as the sole overriding value system.

It might be that when we criticize rationality, we have this implicit goal of some kind of influence in mind. We expect others to care about influence, at least a little bit, if they really thought it through. If that potential influence is big enough, we might even expect it to dominate other concerns (hence the evangelical zeal of effective altruism).

I don't think influence is inherently normative. In many cases though, I think it is reasonable to expect relevant agents to care about their influence. Agents which do not care even instrumentally about effecting the wider world probably will not predominate. They will probably be bystanders and NPCs - and by their own lights, this is fine. 

However, to achieve relevance, it is probably enough to take the first-person view, but to care about one's experiences much more than one's actions-for-their-own-sake, which is a synthesis of consequentialism and self-centeredness. This condition is already strong enough to rule out Batman, but not hedonists or wireheaders. 

  1. ^

    See Dostoevsky's "Notes from the Underground."

  2. ^

    In the sense of genesis.

New Comment
1 comment, sorted by Click to highlight new comments since:

Agents which do not care even instrumentally about effecting the wider world probably will not predominate. They will probably be bystanders and NPCs - and by their own lights, this is fine.


I would point your attention to Rucker's character Sta-Hi Mooney, from the Ware series. An inattentive reader of Software might consider this character a bland hedonist, and in the first sense he is. But through the novel it is revealed that he is very engaged in a Dionysian manner, the disruption of rationality provided by drugs gives him an awareness that is not available to "NPCs," mere bystanders, or even the hyper-rational inventor of AI, Cobb Anderson. We see a Dionysian kind of character also in Fear And Loathing, where only Raul Duke can actually discern the mad American Dream. In a sense these characters are destructive, tearing down common illusions that we all take for granted, and in the case of Sta-Hi, we have it double in his destruction of the Big Boppers.

Rationalism or Apollonian thought is less capable of "play," and I do not mean that in the sense that rational people aren't enjoying themselves. Rather, play in the sense of Derrida where we are stressing the seemingly unlimited permutations within a limited system. This working out of a minmaxing strategy for making sense of the world leads to a very narrow and deterministic style, Narrative.

Apollonian thought is insidious, one may not even notice its narrative building blocks. This hunch that a Rationalist is better prepared for "instrumentally effecting the wider world" cannot be true, as the whole affair is bent towards the crystallization of ideas into more beautiful, solid, or eloquent terms. This is the opposite of transformative action. Ultimately, the destructive impulse of the Dionysian is a perhaps final reflection upon a thought and not a destruction as it poses, and what was once their new paradigm instantly begins to itself crystallize as rules of how to best play the newly-altered game arise. So its destructive mandate is not an end in itself, but a new beginning.

This dynamic is very evident in a lot of popular fiction, as well. When one finds oneself in a lose-lose situation, I think the slightly crazy and compulsive Captain Kirk is in order over rationalist Spock, the Vulcan most often convinced of his own narratives and a total defeatist. Contrast Gandalf with Saruman, the wandering unserious wizard who quite irrationally let the world's fate hang in the hands of a few very hedonistic hobbits rather than the very logical Saruman who wished to build up his own power to betray Sauron.

Curated and popular this week