Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

g_pepper comments on Magical Categories - Less Wrong

24 Post author: Eliezer_Yudkowsky 24 August 2008 07:51PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (129)

Sort By: Old

You are viewing a single comment's thread. Show more comments above.

Comment author: BrianPansky 14 November 2016 11:14:55PM *  0 points [-]

But, what if two different people have two conflicting desires? How do we objectively find the ethical resolution to the conflict?

Basically: game theory.

In reality, I'm not sure there ever are precise conflicts of true foundational desires. Maybe it would help if you had some real example or something. But the best choice for each party will always be the one that maximizes their chances of satisfying their true desire.

Comment author: g_pepper 15 November 2016 05:13:59AM 0 points [-]

I was surprised to hear that you doubt that there are ever conflicts in desires. But, since you asked, here is an example:

A is a sadist. A enjoys inflicting pain in others. A really wants to hurt B. B wishes not to be hurt by A. (For the sake of argument, lets suppose that no simulation technology is available that would allow A to hurt a virtual B, and that A can be reasonably confident that A will not be arrested and brought to trial for hurting B.)

In this scenario, since A and B have conflicting desires, how does a system that defines objective goodness as that which will satisfy desires resolve the conflict?

Comment author: BrianPansky 23 November 2016 05:51:08AM 0 points [-]

I was surprised to hear that you doubt that there are ever conflicts in desires.

Re-read what I said. That's not what I said.

First get straight: good literally objectively does mean desirable. You can't avoid that. Your question about conflict can't change that (thus it's a red herring).

As for your question: I already generally answered it in my previous post. Use Game theory. Find the actions that will actually be best for each agent. The best choice for each party will always be the one that maximizes their chances of satisfying their true desires.

I might finish a longer response to your specific example, but that takes time. For now, Richard Carrier's Goal Theory Update probably covers a lot of that ground.

http://richardcarrier.blogspot.ca/2011/10/goal-theory-update.html

Comment author: CCC 23 November 2016 08:57:23AM 1 point [-]

First get straight: good literally objectively does mean desirable.

It does not.

Wiktionary states that it means "Acting in the interest of good; ethical." (There are a few other definitions, but I'm pretty sure this is the right one here). Looking through the definitions of 'ethical', I find "Morally approvable, when referring to an action that affects others; good. " 'Morally' is defined as "In keeping of requirements of morality.", and 'morality' is "Recognition of the distinction between good and evil or between right and wrong; respect for and obedience to the rules of right conduct; the mental disposition or characteristic of behaving in a manner intended to produce morally good results. "

Nowhere in there do I see anything about "desirable" - it seems to simplify down to "following a moral code". I therefore suspect that you're implicitly assuming a moral code which equates "desirable" with "good" - I don't think that this is the best choice of a moral code, but it is a moral code that I've seen arguments in favour of before.

But, importantly, it's not the only moral code. Someone who follows a different moral code can easily find something that is good but not desirable; or desirable but not good.

Comment author: g_pepper 25 November 2016 06:05:55PM *  0 points [-]

I was surprised to hear that you doubt that there are ever conflicts in desires.

Re-read what I said. That's not what I said.

Right. You said:

In reality, I'm not sure there ever are precise conflicts of true foundational desires.

Do you have an objective set of criteria for differentiating between true foundational desires and other types of desires? If not, I wonder if it is really useful to respond to an objection arising from the rather obvious fact that people often have conflicting desires by stating that you doubt that true foundational desires are ever in precise conflict.

First get straight: good literally objectively does mean desirable.

As CCC has already pointed out, no, it is not apparent that (morally) good and desirable are the same thing. I won’t spend more time on this point since CCC addressed it well.

Your question about conflict can't change that (thus it's a red herring).

The issue that we are discussing is objective morals. Your equating goodness and desirability leads (in my example of the sadist) A to believe that hurting B is good, and B to believe that hurting B is not good. But moral realism holds that moral valuations are statements that are objectively true or false. So, conflicting desires is not a red herring, since conflicting desires leads (using your criterion) to subjective moral evaluations regarding the goodness of hurting B. Game theory on the other hand does appear to be a red herring – no application of game theory can change the fact that A and B differ regarding the desirability of hurting B.

One additional problem with equating moral goodness with desirability is that it leads to moral outcomes that are in conflict with most people’s moral intuitions. For example, in my example of the sadist A desires to hurt B, but most people’s moral intuition would say that A hurting B just because A wants to hurt B would be immoral. Similarly, rape, murder, theft, etc., could be considered morally good by your criterion if any of those things satisfied a desire. While conflicting with moral intuition does not prove that your definition is wrong, it seems to me that it should at a minimum raise a red flag. And, I think that the burden is on you to explain why anyone should reject his/her moral intuition in favor of a moral criterion that would adjudge theft, rape and murder to be morally good if they satisfy a true desire.

Comment author: TheAncientGeek 26 November 2016 08:29:28AM 0 points [-]

First get straight: good literally objectively does mean desirable.

It's not at all clear that morally good means desirable. The idea that the good is the desirable gets what force it has from the fact that "good" has a lot of nonmoral meanings. Good ice cream is desirable ice cream, but what's that got to do with ethics?

Comment author: entirelyuseless 26 November 2016 05:15:20PM 0 points [-]

Morally good means what it is good to do. So there is something added to "good" to get morally good -- namely it is what it is good all things considered, and good to do, as opposed to good in other ways that have nothing to do with doing.

It if it would be good to eat ice cream at the moment, eating ice cream is morally good. And if it would be bad to eat ice cream at the moment, eating ice cream is morally bad.

But when you say "good ice cream," you aren't talking about what it is good to do, so you aren't talking about morality. Sometimes it is good to eat bad ice cream (e.g. you have been offered it in a situation where it would be rude to refuse), and then it is morally good to eat the bad ice cream, and sometimes it is bad to eat good ice cream (e.g. you have already eaten too much), and then it is morally bad to eat the good ice cream.

Comment author: TheAncientGeek 28 November 2016 01:28:06PM 0 points [-]

Morally good means what it is good to do. So there is something added to "good" to get morally good -- namely it is what it is good all things considered, and good to do, as opposed to good in other ways that have nothing to do with doing.

That's a theory of what "morally" is adding to "good". You need to defend it against alternatives, rather than stating it as if it were obvious.

It if it would be good to eat ice cream at the moment, eating ice cream is morally good.

Are you sure? How many people agree with that? Do you have independent evidence , or are you just following through the consequences of your assumptions (ie arguing in circles)?

Comment author: entirelyuseless 28 November 2016 03:51:37PM 0 points [-]

I think most people would say that it doesn't matter if you eat ice cream or not, and in that sense they might say it is morally indifferent. However, while I agree that it mainly doesn't matter, I think they are either identifying "non-morally obligatory" with indifferent here, or else taking something that doesn't matter much, and speaking as though it doesn't matter at all.

But I think that most people would agree that gluttony is a vice, and that implies that there is an opposite virtue, which would mean eating the right amount and at the right time and so on. And eating ice cream when it is good to eat ice cream would be an act of that virtue.

Would you agree that discussion about "morally good" is discussion about what we ought to do? It seems to me this is obviously what we are talking about. And we should do things that are good to do, and avoid doing things that are bad to do. So if "morally good" is about what we should do, then "morally good" means something it is good to do.

Comment author: TheAncientGeek 28 November 2016 04:54:36PM *  1 point [-]

I think most people would say that it doesn't matter if you eat ice cream or not, and in that sense they might say it is morally indifferent. However, while I agree that it mainly doesn't matter, I think they are either identifying "non-morally obligatory" with indifferent here, or else taking something that doesn't matter much, and speaking as though it doesn't matter at all.

What is wrong with saying it doesn't matter at all?

But I think that most people would agree that gluttony is a vice, and that implies that there is an opposite virtue, which would mean eating the right amount and at the right time and so on. And eating ice cream when it is good to eat ice cream would be an act of that virtue

That's pretty much changing the subject.

Would you agree that discussion about "morally good" is discussion about what we ought to do?

And we should do things that are good to do, and avoid doing things that are bad to do

I think it is about what we morally ought to do. If you are playing chess, you ought to move the bishop diagonally, but that is again non-moral.

We morally-should do what is morally good, and hedonistically-should do what is hedonotsitcally-good, and so on. These can conflict, so they are not the same.

Comment author: entirelyuseless 29 November 2016 03:32:14AM 0 points [-]

Talking about gluttony and temperance was not changing the subject. Most people think that morally good behavior is virtuous behavior, and morally bad behavior vicious behavior. So that implies that gluttony is morally bad, and temperance morally good. And if eating too much ice cream can be gluttony, then eating the right amount can be temperance, and so morally good.

There is a lot wrong with saying "it doesn't matter at all", but basically you would not bother with eating ice cream unless you had some reason for it, and any reason would contribute to making it a good thing to do.

I disagree completely with your statements about should, which do not correspond with any normal usage. No one talks about "hedonistically should."

To reduce this to its fundamentals:

"I should do something" means the same thing as "I ought to do something", which means the same thing as "I need to do something, in order to accomplish something else."

Now if we can put whatever we want for "something else" at the end there, then you can have your "hedonistically should" or "chess playing should" or whatever.

But when we are talking about morality, that "something else" is "doing what is good to do." So "what should I do?" has the answer "whatever you need to do, in order to be doing something good to do, rather than something bad to do."

Comment author: TheAncientGeek 05 December 2016 03:55:20PM 0 points [-]

Talking about gluttony and temperance was not changing the subject. Most people think that morally good behavior is virtuous behavior, and morally bad behavior vicious behavior. So that implies that gluttony is morally bad, and temperance morally good. And if eating too much ice cream can be gluttony, then eating the right amount can be temperance, and so morally good.

It's changing the subject because you are switching from an isolated act to a pattern of behaviour.

There is a lot wrong with saying "it doesn't matter at all",

Such as?

but basically you would not bother with eating ice cream unless you had some reason for it, and any reason would contribute to making it a good thing to do.

You are using good to mean morally good again.

I disagree completely with your statements about should, which do not correspond with any normal usage. No one talks about "hedonistically should."

You can't infer the non-existence of a distinction from the fact that it is not regularly marked in ordinary language.

"Jade is an ornamental rock. The term jade is applied to two different metamorphic rocks that are composed of different silicate minerals:

Nephrite consists of a microcrystalline interlocking fibrous matrix of the calcium, magnesium-iron rich amphibole mineral series tremolite (calcium-magnesium)-ferroactinolite (calcium-magnesium-iron). The middle member of this series with an intermediate composition is called actinolite (the silky fibrous mineral form is one form of asbestos). The higher the iron content, the greener the colour.
Jadeite is a sodium- and aluminium-rich pyroxene. The precious form of jadeite jade is a microcrystalline interlocking growth of jadeite crystals.""

"I should do something" means the same thing as "I ought to do something", which means the same thing as "I need to do something, in order to accomplish something else."

So you say. Actually, the idea that ethical claims can be cashed out as hypotheticals is quite contentious.

Now if we can put whatever we want for "something else" at the end there, then you can have your "hedonistically should" or "chess playing should" or whatever.

But when we are talking about morality, that "something else" is "doing what is good to do." So "what should I do?" has the answer "whatever you need to do, in order to be doing something good to do, rather than something bad to do."

Back to the usual problem. What you morally-should do is whatever you need to do, in order to be doing something morally good, is true but vacuous. . What you morally-should do is whatever you need to do, in order to be doing something good is debatable.

Comment author: rkyeun 26 December 2017 10:28:03AM *  0 points [-]

I would be very surprised to find that a universe whose particles are arranged to maximize objective good would also contain unpaired sadists and masochists. You seem to be asking a question of the form, "But if we take all the evil out of the universe, what about evil?" And the answer is "Good riddance." Pun intentional.

Comment author: g_pepper 26 December 2017 11:08:26PM *  0 points [-]

I would be very surprised to find that a universe whose particles are arranged to maximize objective good would also contain unpaired sadists and masochists.

The problem is that neither you nor BrianPansky has proposed a viable objective standard for goodness. BrianPansky said that good is that which satisfies desires, but proposed no objective method for mediating conflicting desires. And here you said “Do remember that your thoughts and preference on ethics are themselves an arrangement of particles to be solved” but proposed no way for resolving conflicts between different people’s ethical preferences. Even if satisfying desires were an otherwise reasonable standard for goodness, it is not an objective standard, since different people may have different desires. Similarly, different people may have different ethical preferences, so an individual’s ethical preference would not be an objective standard either, even if it were otherwise a reasonable standard.

You seem to be asking a question of the form, "But if we take all the evil out of the universe, what about evil?"

No, I am not asking that. I am pointing out that neither your standard nor BrianPansky’s standard is objective. Therefore neither can be used to determine what would constitute an objectively maximally good universe nor could either be used to take all evil out of the universe, nor even to objectively identify evil.