Eugine_Nier comments on What is Metaethics? - Less Wrong

31 Post author: lukeprog 25 April 2011 04:53PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (550)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eugine_Nier 28 April 2011 01:57:04AM *  0 points [-]

This is consistent with Jablonski's point that "it's all preferences."

In keeping with my physics analogy, saying "it's all preferences" about morality is analogous to saying "it's all opinion" about physics.

Comment author: NMJablonski 28 April 2011 02:19:45AM *  7 points [-]

Clearly there's a group of people who dislike what I've said in this thread, as I've been downvoted quite a bit.

I'm not perfectly clear on why. My only position at any point has been this:

I see a universe which contains intelligent agents trying to fulfill their preferences. Then I see conversations about morality and ethics talking about actions being "right" or "wrong". From the context and explanations, "right" seems to mean very different things. Like:

"Those actions which I prefer" or "Those actions which most agents in a particular place prefer" or "Those actions which fulfill arbitrary metric X"

Likewise, "wrong" inherits its meaning from whatever definition is given for "right". It makes sense to me to talk about preferences. They're important. If that's what people are talking about when they discuss morality, then that makes perfect sense. What I do not understand is when people use the words "right" or "wrong" independently of any agent's preferences. I don't see what they are referring to, or what those words even mean in that context.

Does anyone care to explain what I'm missing, or if there's something specific I did to elicit downvotes?

Comment author: wedrifid 28 April 2011 03:29:55AM 3 points [-]

Does anyone care to explain what I'm missing, or if there's something specific I did to elicit downvotes?

You signaled disagreement with someone about morality. What did you expect? :)

Comment author: NMJablonski 28 April 2011 03:42:40AM 1 point [-]

Your explanation is simple and fits the facts!

I like it :)

Comment author: Perplexed 01 May 2011 06:11:16PM 2 points [-]

What I do not understand is when people use the words "right" or "wrong" independently of any agent's preferences. I don't see what they are referring to, or what those words even mean in that context.

Does anyone care to explain what I'm missing, or if there's something specific I did to elicit downvotes?

I don't know anything about downvotes, but I do think that there is a way of understanding 'right' and 'wrong' independently of preferences. But it takes a conceptual shift.

Don't think of morality as a doctrine guiding you as to how to behave. Instead, imagine it as a doctrine teaching you how to judge the behavior of others (and to a lesser extent, yourself).

Morality teaches you when to punish and reward (and when to expect punishment and reward). It is a second-order concept, and hence not directly tied to preferences.

Comment author: XiXiDu 01 May 2011 06:39:55PM 2 points [-]

I do think that there is a way of understanding 'right' and 'wrong' independently of preferences...Morality teaches you when to punish and reward (and when to expect punishment and reward). It is a second-order concept, and hence not directly tied to preferences.

Sociology? Psychology? Game theory? Mathematics? What does moral philosophy add to the sciences that is useful, that helps us to dissolve confusion and understand the nature of reality?

Comment author: Perplexed 01 May 2011 08:49:13PM 3 points [-]

What does moral philosophy add to the sciences that is useful, that helps us to dissolve confusion and understand the nature of reality?

Moral philosophy, like all philosophy, does nothing directly to illuminate the nature of reality. What it does is to illuminate the nature of confusion.

How does someone who thinks that 'morality' is meaningless discuss the subject with someone who attaches meaning to the word? Answer: They talk to each other carefully and respectfully.

What do you call the subject matter of that discussion? Answer: Metaethics.

What do you call success in this endeavor? Answer: "Dissolving the confusion".

Comment author: XiXiDu 02 May 2011 08:41:12AM *  1 point [-]

Moral philosophy, like all philosophy, does nothing directly to illuminate the nature of reality. What it does is to illuminate the nature of confusion.

Moral philosophy does not illuminate the nature of confusion, it is the confusion. I am asking, what is missing and what confusion is left if you disregard moral philosophy and talk about right and wrong in terms of preferences?

Comment author: Perplexed 02 May 2011 03:26:22PM 2 points [-]

I'm tempted to reply that what is missing is the ability to communicate with anyone who believes in virtue ethics or deontological ethics, and therefore doesn't see how preferences are even involved. But maybe I am not understanding your point.

Perhaps an example would help. Suppose I say, "It is morally wrong for Alice to lie to Bob." How would you analyze that moral intuition in terms of preferences. Whose preferences are we talking about here? Alice's, Bob's, mine, everybody else's? For comparison purposes, also analyze the claim "It is morally wrong for Bob to strangle Alice."

Comment author: XiXiDu 02 May 2011 05:13:26PM *  4 points [-]

"It is morally wrong for Alice to lie to Bob."

Due to your genetically hard-coded intuitions about appropriate behavior within groups of primates, your upbringing, cultural influences, rational knowledge about the virtues of truth-telling and preferences involving the well-being of other people, you feel obliged to influence the intercourse between Alice and Bob in a way that persuades Alice to do what you want, without feeling inappropriately influenced by you, by signaling your objection to certain behaviors as an appeal to the order of higher authority .

"It is morally wrong for Bob to strangle Alice."

If you say, "I don't want you to strangle Alice.", Bob might reply, "I don't care what you want!".

If you say, "Strangling Alice might have detrimental effects on your other preferences.", Bob might reply, "I assign infinite utility to the death of Alice!" (which might very well be the case for humans in a temporary rage).

But if you say, "It is morally wrong to strangle Alice.", Bob might get confused and reply, "You are right, I don't want to be immoral!". Which is really a form of coercive persuasion. Since when you say, "It is morally wrong to strangle Alice.", you actually signal, "If you strangle Alice you will feel guilty.". It is a manipulative method that might make Bob say, "You are right, I don't want to be immoral!", when what he actually means is, "I don't want to feel guilty!".

Primates don't like to be readily controled by other primates. To get them to do what you want you have to make them believe that, for some non-obvious reason, they actually want to do it themselves.

Comment author: Perplexed 02 May 2011 07:19:57PM 1 point [-]

This sounds like you are trying to explain-away the phenomenon, rather than explain it. At the very least, I would think, such a theory of morality needs to make some predictions or explain some distinctions. For example, what is it about the situation that causes me to try to influence Alice and Bob using moral arguments in these cases, whereas I use other methods of influence in other cases?

Comment author: XiXiDu 03 May 2011 09:07:24AM 3 points [-]

For example, what is it about the situation that causes me to try to influence Alice and Bob using moral arguments in these cases, whereas I use other methods of influence in other cases?

Complex influences, like your culture and upbringing.That's also why some people don't say that it is morally wrong to burn a paperback book while others are outraged by the thought. And those differences and similarities can be studied, among other fields, in terms of cultural anthropology and evolutionary psychology.

It needs a multidisciplinary approach to tackle such questions. But moral philosophy shouldn't be part of the solution because it is largely mistaken about cause and effect. Morality is an effect of our societal and cultural evolution, shaped by our genetically predisposition as primates living in groups. In this sense moral philosophy is a meme that is part of a larger effect and therefore can't be part of a reductionist explanation of itself. The underlying causes of cultural norms and our use of language can be explained by social and behavioural sciences, applied mathematics like game theory, computer science and linguistics.

Comment author: Amanojack 03 May 2011 12:42:36AM *  -1 points [-]

For example, what is it about the situation that causes me to try to influence Alice and Bob using moral arguments in these cases, whereas I use other methods of influence in other cases?

Guilt works here, for example. (But XiXiDu covered that.) Social pressure also. Veiled threat and warning, too. Signaling your virtue to others as well. Moral arguments are so handy that they accomplish all of these in one blow.

ETA: I'm not suggesting that you in particular are trying to guilt trip people, pressure them, threaten them, or signal. I'm saying that those are all possible explanations as to why someone might prefer to couch their arguments in moral terms: it is more persuasive (as Dark Arts) in certain cases. Though I reject moralist language if we are trying to have a clear discussion and get at the truth, I am not against using Dark Arts to convince Bob not to strangle Alice.

Comment author: Amanojack 03 May 2011 12:37:21AM *  -1 points [-]

Virtue ethicists and deontologists merely express a preference for certain codes of conduct because they believe adhering to these codes will maximize their utility, usually via the mechanism of lowering their time preference.

ETA: And also, as XiXiDu points out, to signal virtuosity.

Comment author: Amanojack 03 May 2011 12:34:24AM 0 points [-]

Upvoted because I strongly agree with the spirit of this post, but I don't think moral philosophy succeeds in dissolving the confusion. So far it has failed miserably, and I suspect that it is entirely unnecessary. That is, I think this is one field that can be dissolved away.

Comment author: XiXiDu 02 May 2011 08:54:09AM 0 points [-]

What do you call the subject matter of that discussion? Answer: Metaethics.

Like if an atheist is talking to a religious person then the subject matter is metatheology?

Comment author: NMJablonski 01 May 2011 06:16:47PM 1 point [-]

imagine it as a doctrine teaching you how to judge the behavior of others (and to a lesser extent, yourself).

Which metrics do I use to judge others?

There has been some confusion over the word "preference" in the thread, so perhaps I should use "subjective value". Would you agree that the only tools I have for judging others are subjective values? (This includes me placing value on other people reaching a state of subjective high value)

Or do you think there's a set of metrics for judging people which has some spooky, metaphysical property that makes it "better"?

Comment author: XiXiDu 01 May 2011 06:50:05PM 4 points [-]

Or do you think there's a set of metrics for judging people which has some spooky, metaphysical property that makes it "better"?

And why would that even matter as long as I am able to realize what I want without being instantly struck by thunder if I desire or do something that violates the laws of morality? If I live a happy and satisfied life of fulfilled preferences but constantly do what is objectively wrong, why exactly would that matter, to whom would it matter and why would I care if I am happy and my preferences are satisfied? Is it some sort of game that I am losing, where those who are the most right win? What if I don't want to play that game, what if I don't care who wins?

Comment author: Perplexed 02 May 2011 12:50:44AM 2 points [-]

If I live a happy and satisfied life of fulfilled preferences but constantly do what is objectively wrong, why exactly would that matter,

Because it harms other people directly or indirectly. Most immoral actions have that property.

to whom would it matter

To the person you harm. To the victim's friends and relatives. To everyone in the society which is kept smoothly running by the moral code which you flout.

and why would I care if I am happy and my preferences are satisfied?

Because you will probably be punished, and that tends to not satisfy your preferences.

Is it some sort of game that I am losing, where those who are the most right win?

If the moral code is correctly designed, yes.

What if I don't want to play that game, what if I don't care who wins?

Then you are, by definition, irrational, and a sane society will eventually lock you up as being a danger to yourself and everyone else.

Comment author: XiXiDu 02 May 2011 08:38:58AM 3 points [-]

Because it harms other people directly or indirectly. Most immoral actions have that property.

Begging the question.

To the person you harm. To the victim's friends and relatives.

Either that is part of my preferences or it isn't.

To everyone in the society which is kept smoothly running by the moral code which you flout.

Either society is instrumental to my goals or it isn't.

Because you will probably be punished, and that tends to not satisfy your preferences.

Game theory? Instrumental rationality? Cultural anthropology?

If the moral code is correctly designed, yes.

If I am able to realize my goals, satisfy my preferences, don't want to play some sort of morality game with agreed upon goals and am not struck by thunder once I violate those rules, why would I care?

Then you are, by definition, irrational...

What is your definition of irrationality? I wrote that if I am happy, able to reach all of my goals and satisfy all of my preferences while constantly violating the laws of morality, how am I irrational?

Comment author: Peterdjones 03 May 2011 11:50:48PM 1 point [-]

What is your definition of irrationality? I wrote that if I am happy, able to reach all of my goals and satisfy all of my preferences while constantly violating the laws of morality, how am I irrational?

It's irrational to think that the evaluative buck stops with your own preferences.

Comment author: nshepperd 04 May 2011 12:07:13AM 2 points [-]

Maybe he doesn't care about the "evaluative buck", which while rather unfortunate, is certainly possible.

Comment author: Peterdjones 04 May 2011 12:36:26AM 2 points [-]

If he doesn't care about rationality, he is still being irrational,

Comment author: Jonathan_Graehl 03 May 2011 01:33:01AM 1 point [-]

Also, what did you mean by

Game theory?

Cultural anthropology?

... in response to "Because you will probably be punished, and that tends to not satisfy your preferences." ?

I think you mean that you should correctly predict the odds and disutility (over your life) of potential punishments, and then act rationally selfishly. I think this may be too computationally expensive in practice, and you may not have considered the severity of the (unlikely event) that you end up severely punished by a reputation of being an effectively amoral person.

Yes, we see lots of examples of successful and happy unscrupulous people in the news. But consider selection effects (that contradiction of conventional moral wisdom excites people and sells advertisements).

Comment author: XiXiDu 03 May 2011 07:54:16AM *  0 points [-]

I meant that we already do have a field of applied mathematics and science that talks about those things, why do we need moral philosophy?

I am not saying that it is a clear cut issue that we, as computationally bounded agents, should abandon moral language, or that we even would want to do that. I am not advocating to reduce the complexity of natural language. But this community seems to be committed to reductionism, minimizing vagueness and the description of human nature in terms of causal chains. I don't think that moral philosophy fits this community.

This community doesn't talk about theology either, it talks about probability and Occam's razor. Why would it talk about moral philosophy when all of it can be described in terms of cultural anthropology, sociology, evolutionary psychology and game theory?

Comment author: timtyler 03 May 2011 09:49:21AM *  0 points [-]

This community doesn't talk about theology either[...]Why would it talk about moral philosophy when all of it can be described in terms of cultural anthropology, sociology, evolutionary psychology and game theory?

It is a useful umbrella term - rather like "advertising".

Comment author: Peterdjones 03 May 2011 12:11:15PM 0 points [-]

Can all of it be described in those terms? Isn't that a philosophical claim?

Comment author: Jonathan_Graehl 03 May 2011 01:28:18AM 1 point [-]

There's nothing to dispute. You have a defensible position.

However, I think most humans have as part of what satisfies them (they may not know it until they try it), the desire to feel righteous, which can most fully be realized with a hard-to-shake belief. For a rational person, moral realism may offer this without requiring tremendous self-delusion. (disclaimer: I haven't tried this).

Is it worth the cost? Probably you can experiment. It's true that if you formerly felt guilty and afraid of punishment, then deleting the desire to be virtuous (as much as possible) will feel liberating. In most cases, our instinctual fears are overblown in the context of a relatively anonymous urban society.

Still, reputation matters, and you can maintain it more surely by actually being what you present yourself as, rather than carefully (and eventually sloppily and over-optimistically) weighing each case in terms of odds of discovery and punishment. You could work on not feeling bad about your departures from moral perfection more directly, and then enjoy the real positive feeling-of-virtue (if I'm right about our nature), as well as the practical security. The only cost then would be lost opportunities to cheat.

It's hard to know who to trust as having honest thoughts and communication on the issue, rather than presenting an advantageous image, when so much is at stake. Most people seem to prefer tasteful hypocrisy and tasteful hypocrites. Only those trying to impress you with their honesty, or those with whom you've established deep loyalties, will advertise their amorality.

Comment author: Perplexed 02 May 2011 12:43:05AM 2 points [-]

I'm claiming that there is a particular moral code which has the spooky game-theoretical property that it produces the most utility for you and for others. That is, it is the metric which is Pareto optimal and which is also a 'fair' bargain.

Comment author: NMJablonski 02 May 2011 02:00:53AM 1 point [-]

So you're saying that there's one single set of behaviors, which, even though different agents will assign drastically different values to the same potential outcomes, balances their conflicting interests to provide the most net utility across the group. That could be true, although I'm not convinced.

Even if it is, though, what the optimal strategy is will change if the net values across the group changes. The only point I have ever tried to make in these threads is that the origin of any applicable moral value must be the subjective preferences of the agents involved.

The reason any agent would agree to follow such a rule set is if you could demonstrate convincingly that such behaviors maximize that agent's utility. It all comes down to subjective values. There exists no other motivating force.

Comment author: Amanojack 02 May 2011 02:36:16AM 0 points [-]

I'm fond of including clarification like, "subjective values (values defined in the broadest possible sense, to include even things like your desire to get right with your god, to see other people happy, to not feel guilty, or even to "be good")."

Some ways I've found to dissolve people's language back to subjective utility:

  1. If someone says something is good, right, bad, or wrong, ask, "For what purpose?"

  2. If someone declares something immoral, unjust, unethical, ask, "So what unhappiness will I suffer as a result?"

But use sparingly, because there is a big reason many people resist dissolving this confusion.

Comment author: Perplexed 02 May 2011 02:29:50AM 0 points [-]

... what the optimal strategy is will change if the net values across the group changes.

True, but that may not be as telling an objection as you seem to think. For example, if you run into someone (not me!) who claims that the entire moral code is based on the 'Golden Rule' of "Do unto others as you would have others do unto you." Tell that guy that moral behavior changes if preferences change. He will respond "Well, duh! What is your point?".

Comment author: NMJablonski 02 May 2011 02:49:25AM *  1 point [-]

Well, duh! What is your point?

There are people who do not recognize this. It was, in fact, my point.

Edit: Hmm, did I say something rude Perplexed?

Comment author: Perplexed 02 May 2011 03:42:10AM 1 point [-]

Hmm, did I say something rude, Perplexed?

Not to me. I didn't downvote, and in any case I was the first to use the rude "duh!", so if you were rude back I probably deserved it. Unfortunately, I'm afraid I still don't understand your point.

Perhaps you were rude to those unnamed people who you suggest "do not recognize this".

Comment author: NMJablonski 02 May 2011 05:53:54PM 3 points [-]

Unfortunately, I'm afraid I still don't understand your point.

I think we may have reached the somewhat common on LW point where we're arguing even though we have no disagreement.

Comment author: AlephNeil 01 May 2011 10:13:13PM *  1 point [-]

Don't think of morality as a doctrine guiding you as to how to behave. Instead, imagine it as a doctrine teaching you how to judge the behavior of others (and to a lesser extent, yourself).

Yes! That's a point that I've repeated so often to so many different people [not on LW, though] that I'd more-or-less "given up" - it began to seem as futile as swatting flies in summer. Maybe I'll resume swatting now I know I'm not alone.

Comment author: Perplexed 02 May 2011 12:56:17AM 1 point [-]

Maybe I'll resume swatting now I know I'm not alone.

Cool! Swat away. Though I'm not particularly happy with the metaphor.

Comment author: Swimmer963 01 May 2011 10:20:37PM 1 point [-]

Don't think of morality as a doctrine guiding you as to how to behave.

This is mainly how I use morality. I control my own actions, not the actions of other people, so for me it makes sense to judge my own actions as good or bad, right or wrong. I can change them. Judging someone else changes nothing about the state of the world unless I can persuade them to act differently.

Comment author: [deleted] 01 May 2011 10:31:01PM 2 points [-]

Judging someone else changes nothing about the state of the world unless I can persuade them to act differently.

Avoiding a person (a) does not (necessarily) persuade them to act differently, but (b) definitely changes the state of the world. This is not a minor nitpicking point. Avoiding people is also called social ostracism, and it's a major way that people react to misbehavior. It has the primary effect of protecting themselves. It often has the secondary effect of convincing the ostracized person to improve their behavior.

Comment author: Swimmer963 01 May 2011 10:33:59PM 1 point [-]

Then I would consider that a case where I could change their behaviour. There are instances where avoiding someone would bother them enough to have an effect, and other cases where it wouldn't.

Comment author: [deleted] 01 May 2011 10:48:26PM 3 points [-]

Avoiding people who misbehave will change the state of the world even if that does not affect their behavior. It changes the world by protecting you. You are part of the world.

Comment author: Perplexed 02 May 2011 12:52:01AM 0 points [-]

it makes sense to judge my own actions as good or bad, right or wrong. I can change them.

Yes, but if you judge a particular action of your own to be 'wrong', then why should you avoid that action? The definition of wrong that I supply solves that problem. By definition if an action is wrong, then it is likely to elicit punishment. So you have a practical reason for doing right rather than doing wrong.

Furthermore, if you do your duty and reward and/or punish other people for their behavior, then they too will have a practical reason to do right rather than wrong.

Before you object "But that is not morality!", ask yourself how you learned the difference between right and wrong.

Comment author: Swimmer963 02 May 2011 11:43:07AM 0 points [-]

ask yourself how you learned the difference between right and wrong.

It's a valid point that I probably learned morality this way. I think that's actually the definition of 'preconventional' morality-it's based on reward/punishment. Maybe all my current moral ideas have roots in that childhood experience, but they aren't covered by it anymore. There are actions that would be rewarded by most of the people around me, but which I avoid because I consider there to be a "better" alternative. (I should be able to think of more examples of this, but I guess one is laziness at work. I feel guilty if I don't do the cleaning and maintenance that needs doing even though everyone else does almost nothing. I also try to follow a "golden rule" that if I don't want something to happen to me, I won't do it to someone else even if the action is socially acceptable amidst my friends and wouldn't be punished.

Comment author: Perplexed 02 May 2011 04:19:34PM *  1 point [-]

I think that's actually the definition of 'preconventional' morality-it's based on reward/punishment.

Ah. Thanks for bringing up the Kohlberg stages - I hadn't been thinking in those terms.

The view of morality I am promoting here is a kind of meta-pre-conventional viewpoint. That is, morality is not 'that which receives reward and punishment', it is instead 'that which (consequentially) ought to receive reward and punishment, given that many people are stuck at the pre-conventional level'.

Comment author: Swimmer963 02 May 2011 04:39:46PM 0 points [-]

'that which (consequentially) ought to receive reward and punishment, given that many people are stuck at the pre-conventional level'.

How many people? I think (I remember reading in my first-year psych textbook) that most adults functionning at a "normal" level in society are at the conventional level: they have internalized whatever moral standards surround them and obey them as rules, rather than thinking directly of punishment or reward. (They may still be thinking indirectly of punishment and reward; a conventionally moral person obeys the law because it's the law and it's wrong to break the law, implicitly because they would be punished if they did.) I'm not really sure how to separate how people actually reason on moral issues, versus how they think they do, and whether the two are often (or ever???) the same thing.

Comment author: Perplexed 02 May 2011 04:55:52PM 0 points [-]

How many people?

How many people are stuck at that level? I don't know.

How many people must be stuck there to justify the use of punishment as deterrent? My gut feeling is that we are not punishing too much unless the good done (to society) by deterrence is outweighed by the evil done (to the 'criminal') by the punishment.

And also remember that we can use carrots as well as sticks. A smile and a "Thank you" provide a powerful carrot to many people. How many? Again, I don't know, but I suspect that it is only fair to add these carrot-loving pre-conventionalists in with the ones who respond only to sticks.

Comment author: Eugine_Nier 28 April 2011 02:45:40AM *  0 points [-]

In keeping with my analogy let's translate your position into the corresponding position on physics:

I see a universe which contains intelligent agents with opinions and/or beliefs. Then I see conversations about physics and reality talking about beliefs being "true" or "false". From the context and explanations, "true" seems to mean very different things. Like:

"My beliefs" or "The beliefs of most agents in a particular place" or "Those beliefs which fulfill arbitrary metric X"

Likewise, "false" inherits its meaning from whatever definition is given for "true". It makes sense to me to talk about opinions and/or beliefs . They're important. If that's what people are talking about when they discuss truth, then that makes perfect sense. What I do not understand is when people use the words "true" or "false" independently of any agent's opinion. I don't see what they are referring to, or what those words even mean in that context.

Do you still agree with the changed version? If not, why not?

(I never realized how much fun it could be to play a chronophone.)

Comment author: NMJablonski 28 April 2011 02:53:02AM *  4 points [-]

Based upon my experiences, physical truths appear to be concrete and independent of beliefs and opinions. I see no cases where "right" has a meaning outside of an agent's preferences. I don't know how one would go about discovering the "rightness" of something, as one would a physical truth.

It is a poor analogy.

Edit: Seriously? I'm not trying to be obstinate here. Would people prefer I go away?

New edit: Thanks wedrifid. I was very confused.

Comment author: Eugine_Nier 28 April 2011 03:14:14AM -2 points [-]

Based upon my experiences, physical truths appear to be concrete and independent of beliefs and opinions.

Please explain what the word "concrete" means independent of anyone's beliefs and opinions.

Comment author: wedrifid 28 April 2011 03:28:24AM 0 points [-]

Seriously? I'm not trying to be obstinate here. Would people prefer I go away?

You're not being obstinate. You're more or less right, at least in the parent. There are a few nuances left to pick up but you are not likely to find them by arguing with Eugine.

Comment author: Marius 28 April 2011 02:47:05AM 1 point [-]

I don't understand what you mean by preferences when you say "intelligent agents trying to fulfill their preferences". I have met plenty of people who were trying to do things contrary to their preferences. Perhaps before you try (or someone tries for you) to distinguish morality from preferences, it might be helpful to distinguish precisely how preferences and behavior can differ?

Comment author: Amanojack 28 April 2011 03:14:24AM *  3 points [-]

Example? I prefer not to stay up late, but here I am doing it. It's not that I'm acting against my preferences, because my current preference is to continue typing this sentence. It's simply that English doesn't differentiate very well between "current preferences"= "my preferences right this moment" and "current preferences"= "preferences I have generally these days."

Seinfeld said it best.

Comment author: Marius 28 April 2011 10:24:49AM 0 points [-]

But I want an example of people acting contrary to their preferences, you're giving one of yourself acting according to your current preferences. Hopefully, NMJablonski has an example of a common action that is genuinely contrary to the actor's preferences. Otherwise, the word "preference" simply means "behavior" to him and shouldn't be used by him. He would be able to simplify "the actions I prefer are the actions I perform," or "morality is just behavior", which isn't very interesting to talk about.

Comment author: Amanojack 01 May 2011 03:20:44PM *  1 point [-]

"This-moment preferences" are synonymous with "behavior," or more precisely, "(attempted/wished-for) action." In other words, in this moment, my current preferences = what I am currently striving for.

Jablonski seems to be using "morality" to mean something more like the general preferences that one exhibits on a recurring basis, not this-moment preferences. And this is a recurring theme: that morality is questions like, "What general preferences should I cultivate?" (to get more enjoyment out of life)

Comment author: Marius 01 May 2011 05:07:19PM *  1 point [-]

Ok, so if I understand you correctly: It is actually meaningful to ask "what general preferences should I cultivate to get more enjoyment out of life?" If so, you describe two types of preference: the higher-order preference (which I'll call a Preference) to get enjoyment out of life, and the lower-order "preference" (which I'll call a Habit or Current Behavior rather than a preference, to conform to more standard usage) of eating soggy bland french fries if they are sitting in front of you regardless of the likelihood of delicious pizza arriving. So because you prefer to save room for delicious pizza yet have the Habit of eating whatever is nearby and convenient, you can decide to change that Habit. You may do so by changing your behavior today and tomorrow and the day after, eventually forming a new Habit that conforms better to your preference for delicious foods.

Am I describing this appropriately? If so, by the above usage, is morality a matter of Behavior, Habit, or Preference?

Comment author: Amanojack 01 May 2011 05:28:45PM 3 points [-]

Sounds fairly close to what I think Jablonski is saying, yes.

Preference isn't the best word choice. Ultimately it comes down to realizing that I want different things at different times, but in English future wanting is sometimes hard to distinguish from present wanting, which can easily result in a subtle equivocation. This semantic slippage is injecting confusion into the discussion.

Perhaps we have all had the experience of thinking something like, "When 11pm rolls around, I want to want to go to sleep." And it makes sense to ask, "How can I make it so that I want to go to sleep when 11pm rolls around?" Sure, I presently want to go to sleep early tonight, but will I want to then? How can I make sure I will want to? Such questions of pure personal long-term utility seem to exemplify Jablonksi's definition of morality.

Comment author: Marius 01 May 2011 05:46:39PM 0 points [-]

ok cool, replying to the original post then.

Comment author: NMJablonski 01 May 2011 05:50:05PM 0 points [-]

Oops, I totally missed this subthread.

Amanojack has, I think, explained my meaning well. It may be useful to reduce down to physical brains and talk about actual computational facts (i.e. utility function) that lead to behavior rather than use the slippery words "want" or "preference".

Comment author: Marius 01 May 2011 05:47:42PM 0 points [-]

What I do not understand is when people use the words "right" or "wrong" independently of any agent's preferences

Assuming Amanojack explained your position correctly, then there aren't just people fulfilling their preferences. There are people doing all kinds of things that fulfill or fail to fulfill their preferences - and, not entirely coincidentally, which bring happiness and grief to themselves or others. So then a common reasonable definition of morality (that doesn't involve the word preferences) is that set of habits that are most likely to bring long-term happiness to oneself and those around one.

Comment author: NMJablonski 01 May 2011 06:01:03PM *  1 point [-]

there aren't just people fulfilling their preferences.

You missed a word in my original. I said that there were agents trying to fulfill their preferences. Now, per my comment at the end of your subthread with Amanojack, I realize that the word "preferences" may be unhelpful. Let me try to taboo it:

There are intelligent agents who assign higher values to some futures than others. I observe them generally making an effort to actualize those futures, but sometimes failing due to various immediate circumstances, which we could call cognitive overrides. What I mean by that is that these agents have biases and heuristics which lead them to poorly evaluate the consequences of actions.

Even if a human sleeping on the edge of a cliff knows that the cliff edge is right next to him, he will jolt if startled by noise or movement. He may not want to fall off the cliff, but the jolt reaction occurs before he is able to analyze it. Similarly, under conditions of sufficient hunger, thirst, fear, or pain, the analytical parts of the agent's mind give way to evolved heuristics.

definition of morality (that doesn't involve the word preferences) is that set of habits that are most likely to bring long-term happiness to oneself and those around one.

If that's how you would like to define it, that's fine. Would you agree then, that the contents of that set of habits is contingent upon what makes you and those around you happy?

Comment author: Marius 02 May 2011 07:43:08PM 0 points [-]

He may not want to fall off the cliff, but the jolt reaction occurs before he is able to analyze it

I suspect it's a matter of degree rather than either-or. People sleeping on the edges of cliffs are much less likely to jot when startled than people sleeping on soft beds, but not 0% likely. The interplay between your biases and your reason is highly complex.

Would you agree then, that the contents of that set of habits is contingent upon what makes you and those around you happy?

Yes; absolutely. I suspect that a coherent definition of morality that isn't contingent on those will have to reference a deity.

Comment author: NMJablonski 03 May 2011 04:16:42AM 1 point [-]

We are, near as I can tell, in perfect agreement on the substance of this issue. Aumann would be proud. :)

Comment author: Peterdjones 28 April 2011 01:17:53PM 0 points [-]

Clearly there's a group of people who dislike what I've said in this thread, as I've been downvoted quite a bit.

Same here.

"Those actions which I prefer" or "Those actions which most agents in a particular place prefer" or "Those actions which fulfill arbitrary metric X"

It doesn't mean any of those things, since any of them can be judged wrong.

Likewise, "wrong" inherits its meaning from whatever definition is given for "right". It makes sense to me to talk about preferences. They're important. If that's what people are talking about when they discuss morality, then that makes perfect sense.

Morality is about having the right preferences, as rationality is about having true beliefs.

What I do not understand is when people use the words "right" or "wrong" >independently of any agent's preferences. I don't see what they are referring to, or >what those words even mean in that context.

Do you think the sentence "there are truths no-one knows" is meaningful?

Comment author: NMJablonski 28 April 2011 04:41:51PM *  0 points [-]

Morality is about having the right preferences, as rationality is about having true beliefs.

I understand what it would mean to have a true belief, as truth is noticeably independent of belief. I can be surprised, and I can anticipate. I have an understanding of a physical world of which I am part, and which generates my experiences.

It does not make any sense for there to be some "correct" preferences. Unlike belief, where there is an actual territory to map, preferences are merely a byproduct of the physical processes of intelligence. They have no higher or divine purpose which demands certain preferences be held. Evolution selects for those which aid survival, and it doesn't matter if survival means aggression or cooperation. The universe doesn't care.

I think you and other objective moralists in this thread suffer from extremely anthropocentric thinking. If you rewind the universe to a time before there are humans, in a time of early expansion and the first formation of galaxies, does there exist then the "correct" preferences that any agent must strive to discover? Do they exist independent of what kinds of life evolve in what conditions?

If you are able to zoom out of your skull, and view yourself and the world around you as interesting molecules going about their business, you'll see how absurd this is. Play through the evolution of life on a planetary scale in your mind. Be aware of the molecular forces at work. Run it on fast forward. Stop and notice the points where intelligence is selected for. Watch social animals survive or die based on certain behaviors. See the origin of your own preferences, and why they are so different from some other humans.

Objective morality is a fantasy of self-importance, and a hold-over from ignorant quasi-religious philosophy which has now cloaked itself in scientific terms and hides in university philosophy departments. Physics is going to continue to play out. The only agents who can ever possibly care what you do are other physical intelligences in your light cone.

Comment author: Vladimir_Nesov 28 April 2011 05:19:27PM 2 points [-]

For the record, I think in this thread Eugine_Nier follows a useful kind of "simple truth", not making errors as a result, while some of the opponents demand sophistication in lieu of correctness.

Comment author: NMJablonski 28 April 2011 05:29:16PM 1 point [-]

I think we're demanding clarity and substance, not sophistication. Honestly I feel like one of the major issues with moral discussions is that huge sophisticated arguments can emerge without any connection to substantive reality.

I would really appreciate it if someone would taboo the words "moral", "good", "evil", "right", "wrong", "should", etc. and try to make the point using simpler concepts that have less baggage and ambiguity.

Comment author: Vladimir_Nesov 28 April 2011 05:32:58PM 1 point [-]

Clarity can be difficult. What do you mean by "truth"?

Comment author: NMJablonski 28 April 2011 05:34:18PM 2 points [-]

I mean it in precisely the sense that The Simple Truth does. Anticipation control.

Comment author: Vladimir_Nesov 28 April 2011 05:39:38PM *  4 points [-]

That's not the point. You must use your heuristics even if you don't know how they work, and avoid demanding to know how they work or how they should work as a prerequisite to being allowed to use them. Before developing technical ideas about what it means for something to be true, or what it means for something to be right, you need to allow yourself to recognize when something is true, or is right.

Comment author: NMJablonski 28 April 2011 05:47:53PM 3 points [-]

I'm sorry, but if we had no knowledge of brains, cognition, and the nature of preference, then sure, I'd use my feelings of right or wrong as much as the next guy, but that doesn't make them objectively true.

Likewise, just because I intuitively feel like I have a time-continuous self, that doesn't make consciousness fundamental.

As an agent, having knowledge of what I am, and what causes my experiences, changes my simple reliance on heuristics to a more accurate scientific exploration of the truth.

Comment author: Vladimir_Nesov 28 April 2011 06:11:52PM 1 point [-]

I'm sorry, but if we had no knowledge of brains, cognition, and the nature of preference, then sure

Just make sure that the particular piece of knowledge you demand is indeed available, and not, say, just the thing you are trying to figure out.

Comment author: NMJablonski 28 April 2011 06:15:10PM *  3 points [-]

(Nod)

I still think it's a pretty simple case here. Is there a set of preferences which all intelligent agents are compelled by some force to adopt? Not as far as I can tell.

Comment author: Peterdjones 28 April 2011 06:36:24PM 0 points [-]

Morality doesn't work like physical law either. Nobody is compelled to be rational, but people who do reason can agree about certain things. That includes moral reasoning.

Comment author: JGWeissman 28 April 2011 02:49:41AM 0 points [-]

saying "it's all preferences" about morality is analogous to saying "it's all opinion" about physics.

No matter what opinions anyone holds about gravity, objects near the surface of the earth not subject to other forces accelerate towards the earth at 9.8 meters per second per second. This is an empirical fact about physics, and we know ways our experience could be different if it were wrong. Do you have an example of a fact about morality, independent of preferences, such that we could notice if it is wrong?

Comment author: Eugine_Nier 28 April 2011 03:08:30AM -1 points [-]

No matter what opinions anyone holds about gravity, objects near the surface of the earth not subject to other forces accelerate towards the earth at 9.8 meters per second per second.

Do you have an example of a fact about morality, independent of preferences,

Killing innocent people is wrong barring extenuating circumstances.

(I'll taboo the "weasel words" innocent and extenuating circumstances as soon as you taboo the "weasel words" near the surface of the earth and not subject to other forces.

such that we could notice if it is wrong?

I'm not sure it's possible for my example to be wrong anymore then its possible for 2+2 to equal 3.

Comment author: NMJablonski 28 April 2011 03:48:59AM *  2 points [-]

What is the difference between:

"Killing innocent people is wrong barring extenuating circumstances"

and

"Killing innocent people is right barring extenuating circumstances"

How do you determine which one is accurate? What observable consequences does each one predict? What do they lead you to anticipate?

Comment author: Eugine_Nier 28 April 2011 03:57:25AM 1 point [-]

How do you determine which one is accurate? What observable consequences does each one predict? What do they lead you to anticipate?

Moral facts don't lead me to anticipate observable consequences, but they do affect the actions I choose to take.

Comment author: [deleted] 28 April 2011 04:03:14AM 2 points [-]

Preferences also do that.

Comment author: Eugine_Nier 28 April 2011 04:06:28AM 1 point [-]

Yes, well opinions also anticipate observations. But in a sense by talking about "observable consequences" your taking advantage of the fact that the meta-theory of science is currently much more developed then the meta-theory of ethics.

Comment author: Peterdjones 28 April 2011 12:57:47PM -1 points [-]

But some preferences can be moral, just as some opinions can be true. There is no automatic entailment from "it is a preference" to "it has nothing to do with ethics".

Comment author: CuSithBell 28 April 2011 03:58:34AM 1 point [-]

The question was - how do you determine what the moral facts are?

Comment author: Eugine_Nier 28 April 2011 04:03:35AM 1 point [-]

Currently, intuition. Along with the existing moral theories, such as they are.

Similar to the way people determined facts about physics, especially facts beyond the direct observation of their senses, before the scientific method was developed.

Comment author: CuSithBell 28 April 2011 04:08:36AM 3 points [-]

Right, and 'facts' about God. Except that intuitions about physics derive from observations of physics, whereas intuitions about morality derive from observations of... intuitions.

You can't really argue that objective morality not being well-defined means that it is more likely to be a coherent notion.

Comment author: Eugine_Nier 28 April 2011 04:11:26AM 3 points [-]

My point is that you can't conclude the notion of morality is incoherent simple because we don't yet have a sufficiently concrete definition.

Comment author: CuSithBell 28 April 2011 04:15:10AM *  5 points [-]

Technically, yes. But I'm pretty much obliged, based on the current evidence, to conclude that it's likely to be incoherent.

More to the point: why do you think it's likely to be coherent?

Comment author: Peterdjones 28 April 2011 12:38:58PM -1 points [-]

Right, and 'facts' about God. Except that intuitions about physics derive from >observations of physics, whereas intuitions about morality derive from observations >of... intuitions.

Which is true, and explains why it is a harder problem than physics, and less progress has been made.

Comment author: wedrifid 28 April 2011 01:00:58PM 0 points [-]

Which is true, and explains why it is a harder problem than physics, and less progress has been made.

I'm not sure I accept either of those claims, explanation or no.

Comment deleted 28 April 2011 04:58:38AM [-]
Comment author: CuSithBell 28 April 2011 03:22:31PM 0 points [-]

Yes, but we've already determined that we don't disagree - unless you think we still do? I was arguing against observing objective (i.e. externally existing) morality. I suspect that you disagree more with Eugine_Nier.

Comment author: JGWeissman 28 April 2011 03:15:32AM 2 points [-]

I'm not sure it's possible for my example to be wrong anymore then its possible for 2+2 to equal 3.

What would it take to convince you your example is wrong?

Note how "2+2=4" has observable consequences:

Suppose I got up one morning, and took out two earplugs, and set them down next to two other earplugs on my nighttable, and noticed that there were now three earplugs, without any earplugs having appeared or disappeared - in contrast to my stored memory that 2 + 2 was supposed to equal 4. Moreover, when I visualized the process in my own mind, it seemed that making XX and XX come out to XXXX required an extra X to appear from nowhere, and was, moreover, inconsistent with other arithmetic I visualized, since subtracting XX from XXX left XX, but subtracting XX from XXXX left XXX. This would conflict with my stored memory that 3 - 2 = 1, but memory would be absurd in the face of physical and mental confirmation that XXX - XX = XX.

Does your example (or another you care to come up with) have observable consequences?

Comment author: prase 28 April 2011 03:26:38PM 1 point [-]

What is weasel-like with "near the surface of the earth"?

Comment author: Eugine_Nier 29 April 2011 12:50:15AM 0 points [-]

In this context, it's as "weasel-like" as "innocent". In the sense that both are fudge factors you need to add to the otherwise elegant statement to make it true.

Comment author: Amanojack 28 April 2011 02:40:14AM 0 points [-]

I don't think you can explicate such a connection, especially not without any terms defined. In fact, it is just utterly pointless to try to develop a theory in a field that hasn't even been defined in a coherent way. It's not like it's close to being defined, either.

For example, "Is abortion morally wrong?" combines about 12 possible questions into it because it has a least that many interpretations. Choose one, then we can study that. I just can't see how otherwise rationality-oriented people can put up with such extreme vagueness. There is almost zero actual communication happening in this thread in the sense of actually expressing which interpretation of moral language anyone is taking. And once that starts happening it will cover way too many topics to ever reach a resolution. We're simply going to have to stop compressing all these disparate-but-subtly-related concepts into a single field, taboo all the moralist language, and hug some queries (if any important ones actually remain).

Comment author: Eugine_Nier 28 April 2011 03:40:43AM 1 point [-]

I don't think you can explicate such a connection, especially not without any terms defined. In fact, it is just utterly pointless to try to develop a theory in a field that hasn't even been defined in a coherent way. It's not like it's close to being defined, either.

In any science I can think of people began developing it using intuitive notions, only being able to come up with definitions after substantial progress had been made.