TheAncientGeek comments on On Terminal Goals and Virtue Ethics - Less Wrong

67 Post author: Swimmer963 18 June 2014 04:00AM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Article Navigation

Comments (205)
Tags: 2 1 system decisions ethics virtue goals terminal

Comments (205)

You are viewing a single comment's thread. Show more comments above.

Comment author: TheAncientGeek 19 June 2014 12:32:32PM -2 points [-]

There's no incoherence in defining "terminal" as "not lowest priority", which is basically what you are saying.

It just not what the word means.

Literally, etymologically, that is not what terminal means. It means maximal, or final. A terminal illness is not an illness that is a bit more serious than some other illness.

It's not even what it usually means on LW. If Clippies goals were terminal in your sense, they would be overridable .....you would be able to talk Clippie out of papercliiping.

What you are talking about is valid, is a thing. If you have any hierarchy of goals, there are some at the bottom, some in the middle, and some at the top. But you need to invent a new word for the middle ones, because, "terminal" doesn't mean "intermediate".

Comment author: gjm 19 June 2014 10:04:14PM 6 points [-]

OK, that makes the source of disagreement clearer.

I agree that "terminal" means "final" (but not that it means "maximal"; that's a different concept). But it doesn't (to me, and I think to others on LW) mean "final" in the sense I think you have in mind (i.e., so supremely important that once you notice it applies you can stop thinking), but in a different sense (when analysing goals or values, asking "so why do I want X?", this is a point at which you can go no further: "well, I just do").

So we're agreed on the etymology: a "terminal" goal or value is one-than-which-one-can-go-no-further. But you want it to mean "no further in the direction of increasing importance" and I want it to mean "no further in the direction of increasing fundamental-ness". I think the latter usage has at least the following two advantages:

  • It's possible that people actually have quite a lot of goals and values that are "terminal" in this sense, including ones that are directly relevant in motivating them in ordinary situations. (Whereas it's very rare to come across a situation in which some goal you have is so comprehensively overriding that you don't have to think about anything else.)
  • This usage of "terminal" is well established on LW. I think its usage here goes back to Eliezer's post called Terminal Values and Instrumental Values from November 2007. See also the LW wiki entry. This is not a usage I have just invented, and I strongly disagree with your statement that "It's not even what it usually means on LW".

The trouble with Clippy isn't that his paperclip-maximizing goal is terminal, it's that that's his only goal.

I'm not sure whether in your last paragraph you're suggesting that I'm using "terminal" to mean "intermediate in importance", but for the avoidance of doubt I am not doing anything at all like that. There are two separate things here that you could call hierarchies, one in terms of importance and one in terms of explanation, and "terminal" refers (in my usage, which I think is also the LW-usual one) only to the latter.

Comment author: Nornagest 20 June 2014 04:49:50PM *  4 points [-]

We can go a step further, actually: "teminal value" and various synonyms are well-established within philosophy, where they usually carry the familiar LW meaning of "something that has value in itself, not as a means to an end".

Comment author: DefectiveAlgorithm 19 June 2014 10:34:06PM 4 points [-]

No. Clippy cannot be persuaded away from paperclipping because maximizing paperclips is its only terminal goal.

Comment author: Ruby 19 June 2014 12:38:54PM 1 point [-]

I feel like there's not much of a distinction being made here between terminal values and terminal goals. I think they're importantly different things.

Comment author: TheAncientGeek 19 June 2014 12:42:40PM *  1 point [-]

Huh?

Comment author: Ruby 19 June 2014 01:04:50PM *  0 points [-]

A goal I set is a state of the world I am actively trying to bring about, whereas a value is something which . . . has value to me. The things I value dictate which world states I prefer, but for either lack of resources or conflict, I only pursue the world states resulting from a subset of my values.

So not everything I value ends up being a goal. This includes terminal goals. For instance, I think that it is true that I terminally value being a talented artist - greatly skilled in creative expression - being so would make me happy in and of itself, but it's not a goal of mine because I can't prioritise it with the resources I have. Values like eliminating suffering and misery are ones which matter to me more, and get translated into corresponding goals to change the world via action.

I haven't seen a definition provided, but if I had to provide one for 'terminal goal' it would be that it's a goal whose attainment constitutes fulfilment of a terminal value. Possessing money is rarely a terminal value, and so accruing money isn't a terminal goal, even if it is intermediary to achieving a world state desired for its own sake. Accomplishing the goal of having all the hungry people fed is the world state which lines up with the value of no suffering, hence it's terminal. They're close, but not quite same thing.

I think it makes sense to possibly not work with terminal goals on a motivational/decision making level, but it doesn't seem possible (or at least likely) that someone wouldn't have terminal values, in the sense of not having states of the world which they prefer over others. [These world-state-preferences might not be completely stable or consistent, but if you prefer the world be one way than another, that's a value.]

Powered by Reddit Powered by Reddit