MugaSofer comments on Train Philosophers with Pearl and Kahneman, not Plato and Kant - Less Wrong

65 Post author: lukeprog 06 December 2012 12:42AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (510)

You are viewing a single comment's thread. Show more comments above.

Comment author: MugaSofer 10 December 2012 06:12:16PM 1 point [-]

And yet a paperclipper has perfectly coherent preferences. Without direct access to some source-of-morality that somehow supersedes mere human ethics, how can we judge our morality except by it's own standards? If you have such a source, it would make an excellent top-level post, of perhaps even a sequence.

Comment author: Peterdjones 10 December 2012 06:30:22PM -2 points [-]

And yet a paperclipper has perfectly coherent preferences

But not coherent moral preferences. It doesn't care if its paperclipping infinges on other's preferences.

how can we judge our morality except by it's own standards?

By coherence, and by its ability to actually be morality, which paperclipping isn't.

Comment author: MugaSofer 11 December 2012 09:07:34AM 1 point [-]

Could you taboo "morality" for me, please? I suspect we are talking at cross-purposes.

Comment author: Peterdjones 11 December 2012 10:49:42AM -1 points [-]

You think paperclipping is morality?

Comment author: MugaSofer 11 December 2012 11:33:47AM *  1 point [-]

As I said, I suspect we are using different definitions of "morality"; could we proceed without using the term?

Comment author: Peterdjones 11 December 2012 11:44:53AM -1 points [-]

It would have helped if you had said why you think we have differnt definitions. I don't think I am asserting anything unsual (as far as the wider world is concerned) when I say morality is principallly about regulating interactions between people so that one persons actions take the interests of affected parties into account. Since, to me, that is a truism, it is hard for me to guess why anyone would demur. Other LWers have defined morality as decision theory, as something that just guides their actions, without necessarily taking others into account. I think that is clearly wrong because it suggests that a highly effective serial killer is "good", since they are maximising their own value. But now I am struggling to guess something you could easily just tell me.

Comment author: MugaSofer 11 December 2012 12:14:20PM 1 point [-]

You stated that there was some way to determine the validity of our ethics - by which I meant the moral preferences humans hold, as distinct from whatever source may have given them to us, be tit prisoner's dilemmas or the will of God - without recourse to those same ethical intuitions.

When challenged on this assertion, you stated that our preferences may be revealed as incoherent by logic; yet, as I pointed out, an agent's preferences may be perfectly coherent without being anything we would regard as "right".

So either there has been some misunderstanding, or ... show us this mysterious method of determining the Rightness of something without recourse to our ethical intuitions.

Comment author: Peterdjones 11 December 2012 01:03:21PM *  0 points [-]

When challenged on this assertion, you stated that our preferences may be revealed as incoherent by logic;

I stated:

By coherence, and by its ability to actually be morality,

emphasis added. Your counterexample was paperclipping, which you say is coherent. My response was:

and by its ability to actually be morality, which paperclipping isn't.

So you still need an example of coherent morlaity that is somehow readically different from ours, showing that coherent morality doens't converge enought to be called objective (or at least EDIT: intersubjective).

Comment author: CCC 11 December 2012 01:36:40PM 3 points [-]

So you still need an example of coherent morlaity that is somehow readically different from ours,

May I refer you to the Chanur series, by C.J. Cherryh? Depicted in that series are several alien species along with alien modes of thought and alien moralities.

Consider for a moment the Kif. The Kif are a race of carnivores; they lack the internal wiring to appreciate emotions (as you and I understand the term "emotion") and eat their food live (the notion of eating dead carrion disgusts them, no matter whether it's been cooked in the meantime). Their terminal value is to maximise a quantity that they refer to as sfik, which has the following properties, among others:

  • If you die, then your sfik instantly goes to negative infinity. Personal survival is of massive value to the Kif. (Survival of others, incidentally, is entirely ignored).
  • Succeeding in any task demonstrates high sfik, with more sfik for more difficult tasks (and more for making it look easy, as compared to narrowly winning).
  • Being able to hold onto something that someone else wants shows greater sfik than the person trying to take it. Conversely, taking something that someone else holds shows greater sfik than the person holding it.
  • Since every Kif will do his utmost to protect his own life, killing another being shows a lot of sfik. This is proportional to the sfik of the person being killed (any high-status Kif is also a high-value target).
  • Of course, a high-sfik Kif gets to that position by being very hard to kill. He may even attract followers, on the basis that he will protect his followers from everybody else (anyone killing one of his followers takes sfik from him in the act, and so he will strive to avoid that). Note that a Kif leader does not promise to protect his followers from himself, and indeed will often kill a follower who has not proved useful (either directly, or by sending him on a suicide mission (and killing him directly if he does not go)).
  • Since they do not share human emotions, they do not grasp the concept of 'friendship'. The closest translation in their language is "temporary-ally-of-convenience".

It's a radically different form of morality; murder is not considered a crime in Kifish society; but it's also coherent (though more complex than paperclipping). I'm not entirely sure what you mean by "it's ability to actually be morality", though; that looks like a circular definition to me.

Comment author: ArisKatsaris 12 December 2012 10:42:58PM *  2 points [-]

I think I am with Peterdjones on this: I don't see how this can be called morality -- it's certainly a set of values, sfik-seeking is certainly something that motivates Kifs, and which they prefer to do; but to call it "morality" in the sense that humans recognize the word is no better than calling it "lust" (though it contains no sex) or "love" (though it contains no element of caring for others).

That it's in their brains and motivates them isn't enough to call it "morality" meaningfully. For it to be called "morality" meaningfully it has to motivate them in roughly the same manner that human morality motivates humans. Baby-eaters and Superhappies were motivated by morality, even if they were vastly different moralities. I don't see anything in the Kifs that could be called morality.

Comment author: Peterdjones 12 December 2012 09:07:29PM *  1 point [-]

I am not convinced that Kif morality is coherent. Looking at it game-theoretically:

There is a set of actions (A) that run a non-infinitessimal risk of infinite loss, ie anything that has a risk of losing ones life.

There is a set of actions (B) that have a non-infintiessimal chance of finite gain.

There is set of inactions that (C) that maintain an equilibrium.

Game theoretically, a Kif should avoid A, and avoid B if it entails A to any. Because losses are infinite and gains finite, it makes no sense to endanger ones life for any putative gain. (Because of the infinity in A, the relative likelihoods of loss and gain don't matter). Kif should either avoid each other, or adopt pacifism (both versions of C). (Kif in fact have much more motivation to be pacifistic than humans). Kif Pacifism if is clearly not C J Cherryh's intention.

The key to whole arguent is the "infinite". Perhaps the "Infinite" is an exageration, or perhaps C J Cherryh is one of those people who thinks infinity is a large finite number. The intended results would follow in that case. Infinity is game changing.

Comment author: Peterdjones 12 December 2012 09:00:10PM 1 point [-]

I have sometimes talked in terms of objective morality, because it seems truer than subjective morality. But morality is not really objective because it will vary with biology, and is of no use to sticks or stones. It is truer still to say that it is intersubjective. Humans are not going to adopt Kif morality because humans are not Kif.

Comment author: MugaSofer 12 December 2012 09:09:08AM 0 points [-]

My first thought on reading this is that a group of Kifish with human morality would eventually rule the world, if not actually wipe out the originals. (My second is "wait, this just a formalization of the standard Evil Bully Race", isn't it?" And my third is "how do they tell how much sfik others have for the purposes of killing them/ taking their stuff?)

Comment author: MugaSofer 12 December 2012 09:03:57AM -1 points [-]

How do you identify morality without simply comparing it to your own intuitions?

Comment author: Peterdjones 12 December 2012 09:52:07AM -1 points [-]

I'm using a kind fo functional role analysis: the role of morality is to regulate the behaviour of each individual to account for the preferences of others. That isn't an intuition in the sense of "men kissing - yeuch!"

Comment author: BerryPick6 10 December 2012 06:38:20PM 0 points [-]

by its ability to actually be morality, which paperclipping isn't.

Why not?

Comment author: Peterdjones 10 December 2012 06:42:57PM -1 points [-]

It doesn't care if its paperclipping infinges on other's preferences.

Comment author: BerryPick6 11 December 2012 12:00:59AM *  0 points [-]

So 'morality'='caring about other people's preferences'?

Comment author: Peterdjones 11 December 2012 02:14:37AM -1 points [-]

Caring about other people's preferences is a necessary but insufficient part of morality.

Comment author: BerryPick6 11 December 2012 03:12:31AM 0 points [-]

So Hedonistic Egoism, for example, doesn't count as morality?

Comment author: Peterdjones 11 December 2012 10:53:38AM -1 points [-]

It may be advertised as such, but I don't have to buy that.