Non-political follow-up to: Ungrateful Hitchhikers (offsite)

Related to: Prices or Bindings?, The True Prisoner's Dilemma

Summary: Situations like the Parfit's Hitchhiker problem select for a certain kind of mind: specifically, one that recognizes that an action can be optimal, in a self-interested sense, even if it can no longer cause any future benefit. A mind that can identify such actions might put them in a different category which enables it to perform them, in defiance of the (futureward) consequentialist concerns that normally need to motivate it. Our evolutionary history has put us through such "Parfitian filters", and the corresponding actions, viewed from the inside, feel like "something we should do", even if we don’t do it, and even if we recognize the lack of a future benefit. Therein lies the origin of our moral intuitions, as well as the basis for creating the category "morality" in the first place.

Introduction: What kind of mind survives Parfit's Dilemma?

Parfit's Dilemma – my version – goes like this: You are lost in the desert and near death. A superbeing known as Omega finds you and considers whether to take you back to civilization and stabilize you. It is a perfect predictor of what you will do, and only plans to rescue you if it predicts that you will, upon recovering, give it $0.01 from your bank account. If it doesn’t predict you’ll pay, you’re left in the desert to die. [1]

So what kind of mind wakes up from this? One that would give Omega the money. Most importantly, the mind is not convinced to withhold payment on the basis that the benefit was received only in the past. Even if it recognizes that no future benefit will result from this decision -- and only future costs will result -- it decides to make the payment anyway.

If a mind is likely to encounter such dilemmas, it would be an advantage to have a decision theory capable of making this kind of "un-consequentialist" decision. And if a decision theory passes through time by being lossily stored by a self-replicating gene (and some decompressing apparatus), then only those that shift to encoding this kind of mentality will be capable of propagating themselves through Parfit's Hitchhiker-like scenarios (call these scenarios "Parfitian filte...