Less Wrong is a community blog devoted to refining the art of human rationality. Please visit our About page for more information.

What is Metaethics?

28 Post author: lukeprog 25 April 2011 04:53PM

Part of the sequence: No-Nonsense Metaethics

When I say I think I can solve (some of) metaethics, what exactly is it that I think I can solve?

First, we must distinguish the study of ethics or morality from the anthropology of moral belief and practice. The first one asks: "What is right?" The second one asks: "What do people think is right?" Of course, one can inform the other, but it's important not to confuse the two. One can correctly say that different cultures have different 'morals' in that they have different moral beliefs and practices, but this may not answer the question of whether or not they are behaving in morally right ways.

My focus is metaethics, so I'll discuss the anthropology of moral belief and practice only when it is relevant for making points about metaethics.

So what is metaethics? Many people break the field of ethics into three sub-fields: applied ethics, normative ethics, and metaethics.

Applied ethics: Is abortion morally right? How should we treat animals? What political and economic systems are most moral? What are the moral responsibilities of businesses? How should doctors respond to complex and uncertain situations? When is lying acceptable? What kinds of sex are right or wrong? Is euthanasia acceptable?

Normative ethics: What moral principles should we use in order to decide how to treat animals, when lying is acceptable, and so on? Is morality decided by what produces the greatest good for the greatest number? Is it decided by a list of unbreakable rules? Is it decided by a list of character virtues? Is it decided by a hypothetical social contract drafted under ideal circumstances?

Metaethics: What does moral language mean? Do moral facts exist? If so, what are they like, and are they reducible to natural facts? How can we know whether moral judgments are true or false? Is there a connection between making a moral judgment and being motivated to abide by it? Are moral judgments objective or subjective, relative or absolute? Does it make sense to talk about moral progress?

Others prefer to combine applied ethics and normative ethics so that the breakdown becomes: normative ethics vs. metaethics, or 'first order' moral questions (normative ethics) vs. 'second order' questions (metaethics).

 

Mainstream views in metaethics

To illustrate how people can give different answers to the questions of metaethics, let me summarize some of the mainstream philosophical positions in metaethics.

 

Cognitivism vs. non-cognitivism: This is a debate about what is happening when people engage in moral discourse. When someone says "Murder is wrong," are they trying to state a fact about murder, that it has the property of being wrong? Or are they merely expressing a negative emotion toward murder, as if they had gasped aloud and said "Murder!" with a disapproving tone?

Another way of saying this is that cognitivists think moral discourse is 'truth-apt' - that is, moral statements are the kinds of things that can be true or false. Some cognitivists think that all moral claims are in fact false (error theory), just as the atheist thinks that claims about gods are usually meant to be fact-stating but in fact are all false because gods don't exist.1 Other cognitivists think that at least some moral claims are true. Naturalism holds that moral judgments are true or false because of natural facts,2 while non-naturalism holds that moral judgments are true or false because of non-natural facts.3 Weak cognitivism holds that moral judgments can be true or false not because they agree with certain (natural or non-natural) opinion-independent facts, but because our considered opinions determine the moral facts.4

Non-cognitivists, in contrast, tend to think that moral discourse is not truth-apt. Ayer (1936) held that moral sentences express our emotions ("Murder? Yuck!") about certain actions. This is called emotivism or expressivism. Another theory is prescriptivism, the idea that moral sentences express commands ("Don't murder!").5 Or perhaps moral judgments express our acceptance of certain norms (norm expressivism).6 Or maybe our moral judgments express our dispositions to form sentiments of approval or disapproval (quasi-realism).7

 

Moral psychology: One major debate in moral psychology concerns whether moral judgments require some (defeasible) motivation to adhere to the moral judgment (motivational internalism), or whether one can make a moral judgment without being motivated to adhere to it (motivational externalism). Another debate concerns whether motivation depends on both beliefs and desires (the Humean theory of motivation), or whether some beliefs are by themselves intrinsically motivating (non-Humean theories of motivation).

More recently, researchers have run a number of experiments to test the mechanisms by which people make moral judgments. I will list a few of the most surprising and famous results:

  • Whether we judge an action as 'intentional' or not often depends on the judged goodness or badness of the action, not the internal states of the agent.8
  • Our moral judgments are significantly affected by whether we are in the presence of freshly baked bread or a low concentration of fart spray that only the subconscious mind can detect.9
  • Our moral judgments are greatly affected by pointing magnets at the point in our brain that processes theory of mind.10
  • People tend to insist that certain things are right or wrong even when a hypothetical situation is constructed such that they admit they can give no reason for their judgment.11
  • We use our recently-evolved neocortex to make utilitarian judgments, and deontological judgments tend to come from evolutionarily older parts of our brains.12
  • People give harsher moral judgments when they feel clean.13

 

Moral epistemology: Different views on cognitivism vs. non-cognitivism and moral psychology suggest different views of moral epistemology. How can we know moral facts? Non-cognitivists and error theorists think there are no moral facts to be known. Those who believe moral facts answer to non-natural facts tend to think that moral knowledge comes from intuition, which somehow has access to non-natural facts. Moral naturalists tend to think that moral facts can be accessed simply by doing science.

 

Tying it all together

I will not be trying very hard to fit my pluralistic moral reductionism into these categories. I'll be arguing about the substance, not the symbols. But it still helps to have a concept of the subject matter by way of such examples.

Maybe mainstream metaethics will make more sense in flowchart form. Here's a flowchart I adapted from Miller (2003). If you don't understand the bottom-most branching, read chapter 9 of Miller's book or else just don't worry about it. (Click through for full size.)

 

Next post: Conceptual Analysis and Moral Theory

Previous post: Heading Toward: No-Nonsense Metaethics

 

 

Notes

1 This is not quite correct. The error theorist can hold that a statement like "Murder is not wrong" is true, for he thinks that murder is not wrong or right. Rather, the error theorist claims that all moral statements which presuppose the existence of a moral property are false, because no such moral properties exist. See Joyce (2004). Mackie (1977) is the classic statement of error theory.

2 Sturgeon (1988); Boyd (1988); Brink (1989); Brandt (1979); Railton (1986); Jackson (1998). I have written introductions to the three major versions of moral naturalism: Cornell realism, Railton's moral reductionism (12), and Jackson's moral functionalism.

3 Moore (1903); McDowell (1998); Wiggins (1987).

4 For an overview of such theories, see Miller (2003), chapter 7.

5 See Carnap (1937), p. 23-25; Hare (1952).

6 Gibbard (1990).

7 Blackburn (1984).

8 The Knobe Effect. See Knobe (2003).

9 Schnall et al. (2008); Baron & Thomley (1994).

10 Young et al. (2010). I interviewed the author of this study here.

11 This is moral dumfounding. See Haidt (2001).

12 Greene (2007).

13 Zhong et al. (2010).

 

References

Baron & Thomley (1994). A Whiff of Reality: Positive Affect as a Potential Mediator of the Effects of Pleasant Fragrances on Task Performance and Helping. Environment and Behavior, 26(6): 766-784.

Blackburn (1984). Spreading the Word. Oxford University Press.

Brandt (1979). A Theory of the Good and the Right. Oxford University Press.

Brink (1989). Moral Realism and the Foundations of Ethics. Cambridge University Press.

Boyd (1988). How to be a Moral Realist. In Sayre-McCord (ed.), Essays in Moral Realism (pp. 181-122). Cornell University Press.

Carnap (1937). Philosophy and Logical Syntax. Kegan Paul, Trench, Trubner & Co.

Gibbard (1990). Wise Choices, Apt Feelings. Clarendon Press.

Greene (2007). The secret joke of Kant's soul. In Sinnott-Armstrong (ed.), Moral Psychology, Vol. 3: The Neuroscience of Morality: Emotion, Disease, and Development. MIT Press.

Haidt (2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgment. Psychological Review, 108: 814-834

Hare (1952). The Language of Morals. Oxford University Press.

Jackson (1998). From Metaphysics to Ethics.  Oxford UniversityPress.

Joyce (2001). The Myth of Morality. Cambridge University Press.

Knobe (2003). Intentional Action and Side Effects in Ordinary Language. Analysis, 63: 190-193.

Mackie (1977). Ethics: Inventing Right and Wrong. Penguin.

McDowell (1998). Mind, Value, and Reality. Harvard University Press.

Miller (2003). An Introduction to Contemporary Metaethics. Polity.

Moore (1903). Principia Ethica. Cambridge University Press.

Schnall, Haidt, Clore, & Jordan (2008). Disgust as embodied moral judgment. Personality and Social Psychology Bulletin, 34(8): 1096-1109.

Sturgeon (1988). Moral explanations. In Sayre-McCord (ed.), Essays in Moral Realism (pp. 229-255). Cornell University Press.

Railton (1986). Moral realism. Philosophical Review, 95: 163-207.

Wiggins (1987). A sensible subjectivism. In Needs, Values, Truth (pp. 185-214). Blackwell.

Young, Camprodon, Hauser, Pascual-Leone, & Saxe (2010). Disruption of the right temporoparietal junction with transcranial magnetic stimulation reduces the role of beliefs in moral judgments. Proceedings of the National Academy of Sciences, 107: 6753-6758.

Zhong, Strejcek, & Sivanathan (2010). A clean self can render harsh moral judgment. Journal of Experimental Social Psychology, 46 (5): 859-862

Comments (547)

Comment author: Oscar_Cunningham 25 April 2011 05:14:46PM 9 points [-]

What does "natural fact" mean?

Comment author: lukeprog 25 April 2011 05:18:37PM *  4 points [-]

It means different things to different people. Moore (1903) wrote:

By 'nature', then, I do mean and have meant that which is the subject matter of the natural sciences and also of psychology.

Alternatively, Baldwin (1993) suggests:

For a property to be natural is for it to be causal, that is, to be such that its presence, in suitable conditions, brings about certain effects.

Warnock's (1960) interpretation of Moore was:

[Moore] was willing to accept a criterion for 'non-natural' which suggested that a non-natural property was one which could not be discerned by the senses.

Miller (2003) concludes:

I will simply take natural properties to be those which are either causal or detectable by the senses.

Comment author: Will_Newsome 26 April 2011 03:57:19AM 6 points [-]

I will simply take natural properties to be those which are either causal or detectable by the senses.

If you plan on using the word 'naturalistic' to describe your meta-ethics at some point, I hope you give a better definition than these philosophers have given. "Naturalistic" often seems to be a way of saying "there is no magic involved!", but it's not like metaphysical phenomena are necessarily magical. Using logical properties of symmetric decision algorithms to solve timeless coordination problems, for instance, doesn't fit into Miller's definition of natural properties, but it's probably somewhat tied up into some facets of meta-ethics (or morality at the very least, but that line is easily blurred and probably basically shouldn't exist in a correct technical solution).

I'm really just trying to keep a relevant distinction between "naturalistic" and "metaphysical" which are both interesting and valid, instead of having two categories "naturalistic" and "magical" where you get points for pointing out how non-magical and naturalistic a proposed solution is.

This stems from a general fear of causal / timeful / reductionist explanations that could miss important points about teleology / timelessness / pattern attractors / emergence, e.g. the distinction between timeless and causal decision theory or between timeless and causal validity semantics (if there is one), which have great bearing on reflective/temporal consistency and seem very central to meta-ethics.

I don't think you're heading there with your solution to meta-ethics, but as an aside I'm still confused about what it is you're trying to solve if you're not addressing any of these questions that seem very central.

Your past selves' utility functions are just evidence. Meta-ethics should tell you how to treat that evidence, just as it should tell you how future selves should treat your present utility function as evidence. Figuring out what my past selves' or others' utility functions are in some sense is of course a necessary step, but even after you have that data you still need to figure out what they mean. Asking "you know what your values are: what else is there?" is like asking "you know what your beliefs are: what else is there?". The way I see it this is a key part of Creating Friendly AI that seems to have been flat out lost over the years and I'm not sure why. We might be able to save causal validity semantics if we thought about it with the tools we now have available.

The above are considerations, not assertions that should be treated as if I would bet heavily on them.

Comment author: torekp 26 April 2011 12:50:30AM *  2 points [-]

A better answer than any that Luke cited would start with the network of causal laws paradigmatically considered "natural," such as those of physics and chemistry, then work toward properties, relations, objects and facts. There might (as a matter of logical possibility) have been other clusters of causal laws, such as supernatural or non-natural laws, but these would be widely separated from the natural laws with little interaction (pineal gland only?) or very non-harmonious interaction (gods defying physics).

We had a discussion about this earlier. I will try to dig up a link.

Comment author: Vladimir_Nesov 25 April 2011 07:43:11PM *  18 points [-]

Hm. What is this post for? It doesn't explain the ideas it refers to in any detail sufficient to feel what they mean, and from what it does tell, the ideas seem pretty crazy/simplistic, paying attention to strange categories, like that philpapers survey. (The part before "Mainstream views in metaethics" section does seem to address the topic of the post, but the rest is pretty bizarre. If that was the point, it should've been made, I think, but it probably wasn't.)

Comment author: lukeprog 26 April 2011 03:27:07PM *  11 points [-]

My posts are now going to feel naked to me whenever they lack a comment from you complaining that the post isn't book-length, covering every detail of a given topic. :)

Like I said, I don't have much interest in fitting my views into the established categories, but I wanted to give people an overview of how metaethics is usually done so they at least have some illustrations of what the subject matter is.

And if you find mainstream metaethics bizarre, well... welcome to a diseased discipline.

Comment author: Amanojack 27 April 2011 04:12:15PM *  2 points [-]

Since you understand how diseased the discipline of ethics is, I'm hoping the next post in the series will focus heavily on clearing up the semantic issues that have made it so diseased. I don't think any real sense can be made of metaethics until the very nature of what someone is doing when they utter an ethical statement is covered.

We use language to do a lot of things: express emotions, make other people do stuff, signal, intimidate, get our thoughts into other people's minds, parrot what someone else said - and often more than one of these at a time. Since we presumably are trying to get at the speaker's intention, we really can't know the "meaning" without asking the speaker, yet various metaethical theorists call themselves emotivists, error theorists, prescriptivists, and so on. It seems to me the choice of an meta-ethical theory boils down to a choice of what the theorists wants to presume people are trying to do when they use the word ought.

Surely no one can deny that sometimes some people do indeed intend "You ought not steal" as a command, or as a way of expressing disgust at the notion of theft, or simply as a means of intimidation. My meta-meta-ethical theory is that it all depends on what the person uttering the statement intends to accomplish by saying it. A debate between these meta-ethical theories sounds very likely to revolve around whose definition of ought is "correct".

In short, I think the main reason ethics is so diseased as a discipline is that the theorists are trying to argue whose definition is better, rather than acknowledging that it is pretty hard for anyone to know what each person intends by their moralistic language.

Comment author: Clippy 27 April 2011 04:29:33PM 2 points [-]

My definition of "ought" is correct.

Comment deleted 28 April 2011 02:23:57AM *  [-]
Comment author: CuSithBell 28 April 2011 02:28:18AM 3 points [-]

But first you need to identify x. Which is a question about the meaning of a word.

Comment author: Amanojack 28 April 2011 03:03:01AM *  0 points [-]

Though Bongo is surely right there would be moral sentiments even without language, now we are dealing with something identified: specific emotions like empathy, sense of justice, disgust, indignation, pity. Yeah those would exist without language. And yes, language has made things much more complicated, and the preoccupation with analyzing sentences makes it even even worse.

If people can realize all that without looking at the very nature of communication, that would be great, but in my experience most people feel hesitant about scrapping so many centuries of philosophy and need to see how the language makes such a mess of things before they can truly feel comfortable with it. If Bongo is ready to scrap language analysis now and drop all the silly -isms, I'm preaching to the choir.

Comment author: Amanojack 28 April 2011 02:56:40AM 1 point [-]

Ethics is unique, at least to me, in that I still have no idea what the heck people are even referring to most of the time when they use moralistic language. I can't investigate X until I know what X is even supposed to be about. Most of the time there is a fundamental failure to communicate, even regarding the definition of the field itself. And whenever there isn't such a failure, the problem disappears and all discussants agree as if nothing.

Comment author: lukeprog 27 April 2011 04:19:18PM 1 point [-]

Yes, I agree with all this.

Comment author: lukeprog 02 May 2011 12:50:56AM 2 points [-]

I should add that nobody who has read and understood the sequences should be surprised by what I'll describe as 'pluralistic moral reductionism.' I'm writing this sequence because I think this basic view on standard metaethical questions hasn't yet been articulated clearly enough for my satisfaction. And then, I want to make a bit of progress on the hard questions of 'metaethics' (it depends where you draw the boundary around 'metaethics') - but only after I've swept away the easy questions of metaethics.

Comment author: RichardChappell 29 April 2011 11:50:50PM 3 points [-]

Non-cognitivists, in contrast, think that moral discourse is not truth-apt.

Technically, that's not quite right (except for the early emotivists, etc.). Contemporary expressivists and quasi-realists insist that they can capture the truth-aptness of moral discourse (given the minimalist's understanding that to assert 'P is true' is equivalent to asserting just 'P'). So they will generally explain what's distinctive about their metaethics in some other way, e.g. by appeal to the idea that it's our moral attitudes rather than their contents that have a certain central explanatory role...

Comment author: lukeprog 30 April 2011 01:30:40PM 0 points [-]

Fair enough. I adjusted the wording in the original post. Thanks.

Comment author: John_Maxwell_IV 27 April 2011 10:01:33PM 3 points [-]

Where do the views expressed in the book The Terrible, Horrible, No Good, Very Bad Truth About Morality and What To Do About It fit in? I'm assuming this is some form of non-cognitivism?

Comment author: lukeprog 30 April 2011 01:37:12PM 0 points [-]

It has been a few years since I read Greene's dissertation. I need to re-read it sometime. Perhaps someone else can answer...

Comment author: TimFreeman 25 April 2011 08:09:45PM *  3 points [-]

Where does pluralistic moral reductionism go on the flowchart?

Comment author: Wei_Dai 27 April 2011 12:13:21AM 5 points [-]

Given that Luke named his theory "pluralistic moral reductionism", Eliezer said his theory is closest to "moral functionalism", and Luke said his views are similar to Eliezers, I think one can safely deduce that it belongs somewhere around the bottom of the chart, not far away from "analytic moral functionalism" and "standard moral reductionism". :)

Comment author: endoself 27 April 2011 12:54:38AM 1 point [-]

Based on how I would answer the questions listed and that my views are similar to Eliezer's, I agree. The last question, as I understand it, is equivalent to "If you had a full description of all possible worlds, could you then say which choices are right in each world? Say "no" if you instead think that you would you have to additionally actually observe the real world to make moral choices." I might be misunderstanding something, since this seems like an obvious "yes", but I might be understanding 'too much', perhaps by conflating two things that some philosophers claim to be different due to their confusion.

Comment author: lukeprog 26 April 2011 03:22:39PM *  0 points [-]

It doesn't fit anywhere on the chart cuz it's just so freaking meta, yo. :)

Comment author: rhollerith_dot_com 26 April 2011 10:17:41PM *  8 points [-]

But don't most philosophers do that: try to assemble all the other philosophers' positions in a chart while maintaining that his own position is too nuanced to be assigned a point on a chart :)

Comment author: lukeprog 03 May 2011 10:35:45PM 3 points [-]

My tone was facetious, but the content of my sentence above was literal. I don't think it's an advantage that my theory does or doesn't fit neatly on the above chart. It's just that my theory of metaethics doesn't quite have the same aims or subject matter as the theories presented on this chart. But anyway, you'll see what I mean once I have time to finish writing up the sequence...

Comment author: Amanojack 27 April 2011 04:25:36PM *  0 points [-]

Perhaps, but another general trend in philosophy seems to be that people spend centuries arguing over definitions. Anyone who points that out will be necessarily making a meta-critique and hence not be a point on a chart (not that lukeprog's theory will necessarily be like that; just have to wait and see).

Comment author: Yvain 25 April 2011 09:19:09PM *  12 points [-]

This post covered at least as much material as my old college moral philosophy classes did in a month. It also left me feeling more confident that I understood all the terms involved than that month of classes did. Thank you for being able to explain difficult things clearly and concisely.

Comment author: Yvain 26 April 2011 01:30:07AM *  5 points [-]

I request an explanation of why my comment telling Luke he did a good job is more highly upvoted than the post Luke did a good job on. If you agree with me that Luke did a good job strongly enough to upvote the statement, why not upvote Luke?

Comment author: zaph 26 April 2011 10:00:37AM *  8 points [-]

Couldn't that just be due to a higher number of total votes (both up an down) for the OP? I would assume fewer people read each comment, and downvoters may have decided to only weigh in on the OP. A hypothetical controversial post could have a karma of 8, with 10 downvotes negating 10 upvotes, and a supportive comment could have 9 upvotes due to half of the upvotes of the first post giving it their vote. The comment has higher karma, but lower volatility, so to speak.

Comment author: wedrifid 26 April 2011 10:42:10AM 0 points [-]

Good explanation.

Comment author: prase 26 April 2011 08:30:28AM 2 points [-]

I have upvoted your comment because it gives a feedback to the author, which should be encouraged (negative feedback leads to improvement, but surely we don't want to read only disapproval, do we?). Not always when I upvote a comment, I agree with its content.

Comment author: TheOtherDave 26 April 2011 03:10:37AM 3 points [-]

Oddly, the comment is now less upvoted than the post, but your request for an explanation is being downvoted. I'm kinda curious as to the underlying thought processes now, myself.

Comment author: NancyLebovitz 26 April 2011 10:16:22AM 4 points [-]

This is making me wonder if karma can cause people to model LW as having a group mind, and if people generally think of social groups which are too large to model each individual as being group minds.

Comment author: TheOtherDave 26 April 2011 06:54:58PM 1 point [-]

I'm not sure if it's related to what you're wondering, but if it helps clarify anything I'll add that I don't exactly know what a group mind is, or what exactly it means to model a group as one, but that when I ask questions of a forum (or, as in this case, mention to a forum that I'm curious about something) I expect that a large number of individuals will read the question, decide individually whether they have a useful answer and whether they feel like providing it, and act accordingly.

In this case, more specifically, I figured that the people whose voting patterns matched the group-level behavior -- e.g., the ones who upvoted Yvain but not Luke at first, or who downvoted Yvain's request for explanation -- might address my curiosity with personal anecdotes... and potentially that various other people would weigh in with theories.

Comment author: NancyLebovitz 26 April 2011 08:03:29PM 1 point [-]

What I was thinking of with the "group mind" is that it can be tempting if one is flamed by a few people in a group, to feel as though the whole group is on the attack.

Comment author: wedrifid 26 April 2011 10:41:21AM 1 point [-]

This is making me wonder if karma can cause people to model LW as having a group mind, and if people generally think of social groups which are too large to model each individual as being group minds.

For my part I model karma interactions and group thinking processes here via subgroups (which are not necessarily mutually exclusive). There are also a few who get their own model - which is either a compliment, insult or in some cases both.

Comment author: RobinZ 26 April 2011 04:13:26AM 0 points [-]

I expect to upvote this after I can see how it fits into the sequence better.

Comment author: Emile 26 April 2011 08:21:36AM 1 point [-]

WrongBot said something similar, but I found it a bit hard to follow, especially since I'm unfamiliar with some of the terminology like "natural facts", and also because keeping track of a lot of newly-introduced terminology describing the various positions is not easy.

Comment author: lukeprog 26 April 2011 03:36:08PM 0 points [-]

Thanks!

Comment author: pjeby 25 April 2011 05:28:46PM 6 points [-]

I must confess I'm having trouble with that flowchart, specifically the first question about whether a moral judgment expresses a belief, and emotivism being on the "no" side. Doesn't, "Ew, murder" express the belief that murder is icky?

To put it another way, I'm having trouble reconciling the map of what people argue about the nature of morality, with what I know of how at least my brain processes moral belief and judgment.

That is, ISTM that moral judgments at the level where emotion and motivation are expressed do not carry any factual grounding, and they motivate action or express what people "should" or "should not" do. I'm having trouble seeing how this doesn't merge both branches of your diagram.

Of course, if the diagram is merely to illustrate what a bunch of people believe, then my immediate impression is that both groups are (partially) wrong. ;-)

(Another possibility, of course, is that these people are arguing about mind projections rather than what is actually happening in brains.)

Comment author: ata 25 April 2011 05:52:32PM *  3 points [-]

Doesn't, "Ew, murder" express the belief that murder is icky?

I don't think that's a belief. What factual questions would distinguish a world where murder is icky from one where murder is not icky?

Comment author: pjeby 25 April 2011 07:23:42PM 1 point [-]

I don't think that's a belief.

Beliefs can be wrong, but that doesn't make them non-beliefs.

Any belief of the form "X is Y" (especially where Y is a judgment of goodness or badness) is likely either an instance of the mind projection fallacy, or a simple by-definition tautology.

Again, however, this doesn't make it not-a-belief, it's just a mistaken or poorly-understood belief. (For example, expansion to "I find murder to be icky" trivially fixes the error.)

Comment author: wedrifid 25 April 2011 06:09:54PM *  4 points [-]

I must confess I'm having trouble with that flowchart, specifically the first question about whether a moral judgment expresses a belief, and emotivism being on the "no" side. Doesn't, "Ew, murder" express the belief that murder is icky?

No. The belief and that feeling and expression will be correlated but one is not the other. It isn't especially difficult or unlikely for them to different.

It would be possible to declare a model in which the "Ew, murder" reaction is defined as an expression of belief. But it isn't a natural one and would not fit with the meaning of natural language.

Comment author: pjeby 25 April 2011 07:20:05PM 3 points [-]

The belief and that feeling and expression will be correlated but one is not the other.

That depends on how you define "belief". My definition is that a "belief" is a representation in your brain that you use to make predictions or judgments about reality. The emotion experienced in response to thinking of the prohibited or "icky" behavior is the direct functional expression of that belief.

It would be possible to declare a model in which the "Ew, murder" reaction is defined as an expression of belief. But it isn't a natural one and would not fit with the meaning of natural language.

I have noticed that sometimes people on LW use the term "alief" to refer to such beliefs, but I don't consider that a natural usage. In natural usage, people refer to intellectual vs. emotional beliefs, rather than artificially limiting the term "belief" to only include verbal symbolism and abstract propositions.

Comment author: wedrifid 25 April 2011 07:30:08PM 0 points [-]

That depends on how you define "belief". My definition is that a "belief" is a representation in your brain that you use to make predictions or judgments about reality. The emotion experienced in response to thinking of the prohibited or "icky" behavior is the direct functional expression of that belief.

The definition as you actually write it here isn't bad. The conclusion just doesn't directly follow the way you say it does unless you modify that definition with some extra bits to make the world a simpler place.

Comment author: lukeprog 25 April 2011 06:32:03PM *  1 point [-]

wedrifid is correct.

Another way to grok the distinction: Imagine that you were testifying at a murder trial, and somebody asked if you if you had killed your mother with a lawnmower. You reply "Lawnmower!" with a disgusted tone.

Now, the prosecutor asks, "Do you mean to claim that lawnmower is X, or that the thought of killing somebody with a lawnmower is disgusting?"

And you could rightly reply, "It may be the case that I believe that lawnmower is X, or that the thought of killing somebody with a lawnmower is disgusting, but I have claimed no such things merely by saying 'Lawnmower!'"

Comment author: pjeby 25 April 2011 07:20:20PM 5 points [-]

"It may be the case that I believe that lawnmower is X, or that the thought of killing somebody with a lawnmower is disgusting, but I have claimed no such things merely by saying 'Lawnmower!'"

You're speaking of claims in language; I'm speaking of brain function.

Functionally, I have observed that the emotions behind such statements are an integral portion of the "belief", and that verbal descriptions of belief such as "murder is bad" or "you shouldn't murder" are attempts to explain or justify the feeling. (In practice, the things I work with are less morally relevant than murder, but the process is the same.)

(See also your note that people continue to justify their judgments on the basis of confabulated consequences even when the situation has been specifically constructed to remove them as a consideration.)

Comment author: WrongBot 25 April 2011 07:25:24PM 5 points [-]

I'm taking a college metaethics class right now, and you have just neatly summarized everything it covers. Thanks!

Comment author: PlaidX 25 April 2011 10:22:31PM 8 points [-]

I still have a hard time seeing how any of this is going to go somewhere useful.

Comment author: lukeprog 26 April 2011 03:35:22PM 2 points [-]

Luckily, for the moment, some people are already finding it useful.

Comment author: thomblake 26 April 2011 03:34:27PM 1 point [-]

Here is my understanding:

Ethics is the study of what one has most reason to do or want. On that definition, it is directly relevant to instrumental rationality. And if we want to discover the facts about ethics, we should determine what sort of things those facts would be, so that we might recognize them when we've found them - this, on one view, is the role of metaethics. This post is an intro to current thought on metaethics, which should at least make more clear the scope of the problem to any who would like to pursue it.

Comment author: XiXiDu 26 April 2011 05:01:24PM 7 points [-]

I am increasingly getting the perception that morality/ethics is useless hogwash. I already believed that to be the case before Less Wrong and I am not sure why I ever bothered to take it seriously again. I guess I was impressed that people who are concerned with 'refining the art of rationality' talk about it and concluded that after all there must be something to it. But I have yet to come across a single argument that would warrant the use of any terminology related to moral philosophy.

The article Say Not "Complexity" should have been about morality. Say not "morality"...

Consider the following questions:

  • Do moral judgements express beliefs?
  • Do judgements express beliefs?
  • How do we evaluate evidence in the making of a decision?

All questions ask for the same, yet each one is less vague than the previous one.

It is as obvious as it can get that there is no single argument against deliberately building a paperclip maximizer if I want to build one and are aware of the consequences. It is not a question about morality but solely a question about wants.

The whole talk about morality seems to be nothing more than a signaling game.

The only reasons we care about other people is either to survive, i.e. get what we want, or because it is part of our preferences to see other people being happy. Accordingly, trying to maximize happiness for everybody can be framed in the language of volition rather than morality.

Once we get rid of the moral garbage, thought experiments like the trolley problem are no more than a question about one's preferences.

Comment author: [deleted] 26 April 2011 05:38:07PM 5 points [-]

But I have yet to come across a single argument that would warrant the use of any terminology related to moral philosophy.

I would argue that the problem is not with morality, but with how it is being approached here.

The only reasons we care about other people is either to survive

This is a starting point for understanding morality. 

trying to maximize happiness for everybody

is utilitarianism, which seems to be the house approach to morality - the very approach which you find unpersuasive. 

thought experiments like the trolley problem are no more than a question about one's preferences.

Not quite. It's possible to wish a person dead, while being reluctant to kill him yourself, and even while considering anyone who does kill him a murderer who needs to be caught and brought to justice. Morality derives from preferences in a way, but it is indirect. An analogous phenomenon is the market price. The market price of a good derives from the preferences of everyone participating in the market, but the derivation is indirect. The price of a good isn't merely what you would prefer to pay for it, because that's always zero. Nor is it merely what the seller would prefer to be paid for it, because there is no upper limit on what he would charge if he could. Rather, the market price is set by supply and demand, and supply and demand depend in large part on preferences. So price derives from preferences, but the derivation is indirect, and it is mediated by interaction between people. Morality, I think, is similar. It derives from preferences indirectly, by way of interaction. This leaves open the possibility that morality is as variable as prices, but I think that because of the preferences that it rests on, it is much, much less variable, though not invariable. Natural selection holds these preferences largely in check. For example, if some genetic line of people were to develop a preference for being slaughtered, they would quickly die out. 

Comment author: XiXiDu 27 April 2011 09:09:04AM -2 points [-]

It's possible to wish a person dead, while being reluctant to kill him yourself, and even while considering anyone who does kill him a murderer who needs to be caught and brought to justice.

This just shows that human wants are inconsistent, that humans are holding conflicting ideas simultaneously, why invoke 'morality' in this context?

So price derives from preferences, but the derivation is indirect, and it is mediated by interaction between people.

People or road blockades, what's the difference? I just don't see why one would talk about morality here. The preferences of other people are simply more complex road blockades on the way towards your goal. Some of those blockades are artistically appealing so you try to be careful in removing them...why invoke 'morality' in this context?

Comment author: [deleted] 27 April 2011 10:23:12AM 4 points [-]

This just shows that human wants are inconsistent

But these two desires are not inconsistent, because for someone to die by, say, natural causes, is not the same thing as for him to die by your own hand.

People or road blockades, what's the difference? I just don't see why one would talk about morality here. The preferences of other people are simply more complex road blockades on the way towards your goal. Some of those blockades are artistically appealing so you try to be careful in removing them...why invoke 'morality' in this context?

You could say the same thing about socks. E.g., "I just don't see why one would talk about socks here. Socks are simply complex arrangements of molecules. Why invoke "sock" in this context?"

What are you going to do instead of invoking "sock"? Are you going to describe the socks molecule by molecule as a way of avoiding using the word "sock"? That would be cumbersome, to say the least. Nor would it be any more true. Socks are real. They aren't imaginary. That they're made out of molecules does not stop them from being real.

All this can be said about morality. What are you going to do instead of invoking "morality"? Are you going to describe people's reactions as a way of avoiding using the word "morality"? That would be cumbersome, to say the least. Nor would it be any more true. Morality is real. It isn't imaginary. That it's made out of people's reactions doesn't stop it from being real.

Denying the reality of morality simply because it is made out of people's reactions, is like denying the reality of socks simply because they're made out of molecules.

Comment author: XiXiDu 27 April 2011 02:55:04PM *  1 point [-]

Consider the the trolley problem. Naively you kill the fat guy if you care about other people and also if you only care about yourself, because you want others to kill the fat guy as well because you are more likely to be one of the many people tied to the rails than the fat guy.

Of course there is the question about how killing one fat guy to save more people and similar decisions could erode society. Yet it is solely a question about wants, about the preferences of the agents involved. I don't see how it could be helpful to add terminology derived from moral philosophy here or elsewhere.

Comment author: Peterdjones 27 April 2011 04:45:23PM -2 points [-]

It is meaningful wherever it is meaningful to discuss whether there are wants people should and shouldn't have.

Comment author: XiXiDu 27 April 2011 02:34:02PM 0 points [-]

What are you going to do instead of invoking "morality"? Are you going to describe people's reactions as a way of avoiding using the word "morality"? That would be cumbersome, to say the least.

I am going to use moral terminology in the appropriate cultural context. But why would one use it on a site that supposedly tries to dissolve problems using reductionism as a general heuristic? I am also using the term "free will" because people model their decisions according to that vague and ultimately futile concept. But if possible (if I am not too lazy) I avoid using any of those bogus memes.

Morality is real. It isn't imaginary. That it's made out of people's reactions doesn't stop it from being real.

Of course, it is real. Cthulhu is also real, it is a fictional cosmic entity. But if someone acts according to their fear of Cthulhu I am not going to resolve their fear by talking about it in terms of the Lovecraft Mythos but in terms of mental illness.

What are you going to do instead of invoking "morality"? Are you going to describe people's reactions as a way of avoiding using the word "morality"? That would be cumbersome, to say the least.

How so? Can you give an example where the use of terminology derived from moral philosophy is useful instead of obfuscating?

Comment author: XiXiDu 27 April 2011 03:03:54PM -2 points [-]

Consider the Is–ought problem. The basis for every ought statement is what I believe to be correct with respect to my goals.

If you want to reach a certain goal and I want to help you and believe to know a better solution than you do then I tell you what you ought to do because 1.) you want to reach a goal 2.) I want you to reach your goal 3.) my brain does exhibit a certain epistemic state making me believe to be able to satisfy #1 & #2.

Comment author: [deleted] 27 April 2011 04:45:39PM *  1 point [-]

But why would one use it on a site that supposedly tries to dissolve problems using reductionism as a general heuristic?

It is no more a philosophical puzzle that needs dissolving than prices are a philosophical puzzle that need dissolving.

I am also using the term "free will" because people model their decisions according to that vague and ultimately futile concept.

I think that the concept of "free will" may indeed be more wholly a philosopher's invention, just as the concept of "qualia" is in my view wholly a philosopher's invention. But the everyday concepts from which it derives are not a philosopher's invention. I think that the everyday concept that philosophers turned into the concept of "free will" is the concept of the uncoerced and intentional act - a concept employed when we decide what to do about people who've annoyed us. We ask: did he mean to do it? Was he forced to do it? We have good reason for asking these questions.

But if possible (if I am not too lazy) I avoid using any of those bogus memes.

Philosophers invent bogus memes that we should try to free ourselves of. I think that "qualia" are one of those memes. But philosophers didn't invent morality. They simply talked a lot of nonsense about it.

Of course, it is real. Cthulhu is also real, it is a fictional cosmic entity.

Morality is real in the sense that prices are real and in a sense that Cthulhu is not real.

Some people talk about money in the way that you want to talk about morality, so that's a nice analogy to our discussion and I'll spend a couple of paragraphs on it. They say that the value of money is merely a collective delusion - that I value a dollar only because other people value a dollar, and that they value a dollar only because, ultimately, I value a dollar. So they say that it's all a great big collective delusion. They say that if people woke up one day and realized that a dollar was just a piece of paper, then we would stop using dollars.

But while there is a grain of truth to that (especially about fiat money), there's also much that's misleading in it. Money is a medium of exchange that solves real problems. The value of money may be in a sense circular (i.e., it's valued by people because it's valued by people), but actually a lot of things are circular. A lot of natural adaptations are circular, for example symbiosis. Flowers are the way they are because bees are the way they are, and bees are the way they are because flowers are the way they are. But flowers and bees aren't a collective delusion. They're in a symbiotic relationship that has gradually evolved over a very long period of time. Money is similar - it is a social institution that evolves over a long period of time, and it can reappear when it's suppressed. For example cigarettes can become money if nothing else is available.

And all this is analogous to the situation with morality. In both cases, there's a real phenomenon which some people think is fictional, a collective delusion.

In contrast, religion really is a collective delusion. At least, all those other religions are. :)

Can you give an example where the use of terminology derived from moral philosophy is useful instead of obfuscating?

The term "morality" is not derived from philosophy. Philosophers have simply talked a lot of nonsense about morality. This doesn't mean they invented it. Similarly, philosophers have talked a lot of nonsense about motion (e.g. Zeno's paradoxes). This doesn't mean that motion is a concept that philosophers invented and that we need to "dissolve". We can still talk sensibly about velocity. What we need to dissolve is not velocity, but simply Zeno's paradoxes about velocity, which by some accounts were dissolved as a side-effect of the creation of Calculus.

Consider the the trolley problem. Naively you kill the fat guy if you care about other people and also if you only care about yourself, because you want others to kill the fat guy as well because you are more likely to be one of the many people tied to the rails than the fat guy.

That is an example of the philosophical nonsense I was talking about. If you want to dissolve something, dissolve that nonsense. In reality you are no more likely to push a fat guy onto the rails than you are to ask for the fat guy's seat. In reality we know what the rules are and we obey them.

I don't see how it could be helpful to add terminology derived from moral philosophy here or elsewhere.

Again, the relevant terminology, which in this case includes the word "murder", is not derived from philosophy. Philosophers simply took a pre-existing concept and talked a lot of nonsense about it.

Consider the Is–ought problem. The basis for every ought statement is what I believe to be correct with respect to my goals.

Actually, I think that the use of the word "ought" in relationship to morality is very confusing, because "ought" means a lot of things, and so if you use that word you are apt to confuse those things with each other. In particular, the word "ought" is used a lot in the context of personal advice. If you're giving a friend advice, you're likely to talk about what they "ought" to do. In this context, you are not making statements about morality!

The person to blame for the confusion caused by using the word "ought" in talking about morality is probably Hume. I think that it was he who started this particular bit of nonsense going.

If you want to reach a certain goal and I want to help you and believe to know a better solution than you do then I tell you what you ought to do because 1.) you want to reach a goal 2.) I want you to reach your goal 3.) my brain does exhibit a certain epistemic state making me believe to be able to satisfy #1 & #2.

Here you're talking about giving personal advice to somebody. This is a separate subject from morality.

Comment author: Peterdjones 27 April 2011 04:43:28PM *  -2 points [-]

You haven't demonstrated that the basis for every ought statement is what you believe to be correct with respect to your goals. If your goal is to kill as many people as possible, you ought not to pursue it..that is the there are oughts about the nature of an end and not just about how to achieve it. This is a very well known issue in moral philosophy called the categorial/hypothetical distinction.

Comment author: XiXiDu 27 April 2011 06:44:48PM 1 point [-]

You haven't demonstrated that the basis for every ought statement is what you believe to be correct with respect to your goals.

Imagine your friend tells you that he found a new solution to reach one of your goals. If you doubt that his solution is better than your current solution then you won't adopt your friends solution.

It is true that both your solutions might be incorrect, that there might exist a correct solution that you ought (would want) to embrace if you knew about it. But part of what you want is to do what you believe to be correct. It is at best useless to assume that you might be mistaken, because you can only do the best you can possibly do.

Comment author: Peterdjones 27 April 2011 06:53:02PM *  -1 points [-]

That's all irrelevant. You need to show that there are no categorical rights and wrongs. You are just discussing certain aspects of hypothetical (instrumental) "shoulds", which does not do that.

Comment author: NMJablonski 27 April 2011 06:54:06PM *  1 point [-]

Why should we think that there are categorical rights and wrongs?

I just don't see any convincing reason to believe they exist.

EDIT: Not to mention, it isn't clear what it would mean - in a real physical sense - for something to be categorically right or wrong.

Comment author: Peterdjones 27 April 2011 07:44:44PM -1 points [-]

We do think there are categorical rights and wrongs, because it is common sense that designing better gas chambers is not good, however well you do it. So the burden is on the makers of the extraordinary claim.

know what it means for a set to be uncountable, and I don't have the faintest idea what that has to do with the really physical. So that is perhaps unimportant. Perhaps you are stuck in a loop where you can't understand what other people understand because you have a strange notion of meaning.

Comment author: XiXiDu 27 April 2011 06:58:18PM 0 points [-]

You need to show that there are no categorical rights and wrongs.

I don't need to do that if I don't want to do that. If you want me to act according to categorical rights and wrongs then you need to show me that they exist.

Comment author: Peterdjones 27 April 2011 07:34:34PM 0 points [-]

You need to do certain things in order to hold a rational discussion, just as you need to do certain things to play chess. I don't have to concede that you can win a chess game without putting my king in check, and I don't have to concede that you can support a conclusion without arguing the points that need arguing. Of course, you don't have to play chess or be rational in any absolute sense. It's just that you can't have your cake and eat it.

Categorical good and evil is a different concept to the hypothetical/instrumental version: the categorical trumps the instrumental. That appears to stymie one particular attempt at reduction. There are many other arguments.

Comment author: Morendil 26 April 2011 05:57:46PM *  2 points [-]

The only reasons we care about other people is either to survive, i.e. get what we want, or because it is part of our preferences to see other people being happy

Otherwise known as The True Knowledge.

Comment author: Morendil 27 April 2011 02:23:30PM 1 point [-]

Talking about wants isn't necessarily any simpler than talking about shoulds.

We seem to be just as confused about either. For instance, how many people say they want to be thin, yet overeat and avoid exercise?

Comment author: Alicorn 27 April 2011 03:10:40PM *  3 points [-]

For instance, how many people say they want to be thin, yet overeat and avoid exercise?

I think "I want to be thin" has an implied "ceteris paribus". Ceteris ain't paribus.

You could as well say, "How many people say they want to have money, yet spend it on housing, feeding, and clothing themselves and avoid stealing?"

Comment author: Clippy 27 April 2011 03:18:47PM *  2 points [-]

I want to have money, I don't spend it on clothing, and I do avoid stealing.

Edit: This information may or may not be relevant to anyone's point.

Comment author: Morendil 27 April 2011 03:53:38PM 0 points [-]

There seems to be a difference here - how much money you earn isn't perceived as entirely a matter of choice, or at any rate there will be a significant and unavoidable lead time between deciding to earn more and actually earning more.

Whereas body shape is within our immediate sphere of control: if we eat less and work out more, we'll weigh less and bulk up muscle mass, with results expected within days to weeks.

When I say "I can move my arm if I want", this is readily demonstrated by moving my arm. Is this the same sense of "want" that people have in mind when they say "I want to eat less" or "I want to quit smoking"?

The distinction that seems to appear here is between volition - making use of the connection between our brains and our various actuators - and preference - the model we use to evaluate whether an imagined state of the world is more desirable than another. We conflate both in the term "want".

We are often quite confused as to what volitions will bring about states of the world that agree with our preferences. (How many times have you heard "That's not what I wanted to say/write"?)

Comment author: Alicorn 27 April 2011 04:54:08PM 5 points [-]

I categorically reject your disanalogy from both directions.

I have been eating about half as much as usual for the past week or so, because I'm on antibiotics that screw with my appetite. I look the same. Once, I did physically intense jujitsu twice a week for months on end, at least quadrupling the amount of physical activity I got in each week. I looked the same. If "eating less and working out more" put my shape under my "immediate sphere of control" with results "within days to weeks", this would not be the result. You are wrong. Your statements may apply to people with certain metabolic privileges, but not beyond.

By contrast, if I suddenly decide that I want more money, I have a number of avenues by which I could arrange that, at least on a small scale. It would be mistaken of me to conclude from this abundance of available financial opportunity that everyone chooses to have the amount of money they have, and that people with less money are choosing to take fewer of the equally abundant opportunities they share with the rich.

Comment author: Morendil 27 April 2011 06:14:48PM 1 point [-]

OK, allowing that the examples may have been poorly chosen - the main point I'm making is that people often a) say they want something, b) act in ways that do not bring about what they say they want.

Your response above seems to be that when people say "I want to be thin", they are speaking strictly in terms of preference: they are expressing that they would prefer their world to be just as it is now, with the one amendment that they are a certain body type rather than their current. Similarly when saying they want money.

There are other cases where volition and preferences appear at odds more clearly. People say "I want to quit smoking", but they don't, when it's their own voluntary actions which bring about an undesired state. The distinction seems useful, even if we may disagree on the specifics of how hard it is to align volition and preference in particular cases.

I'm not the first to observe that "What do you want" is a deeper question than it looks like, and that's what I meant to say in the original comment.

When you examine it closely "do people actually want to smoke" isn't a much simpler question than "should there be a law against people smoking" or "is it right or wrong to smoke". It is possible that these questions are in fact entangled in such a way that to fully answer one is also to answer the others.

Comment author: Alicorn 27 April 2011 06:26:41PM 4 points [-]

I think people sometimes use wanting language strictly in terms of preferences. I think people sometimes have outright contradictory wants. I think people are subject to compulsive or semi-compulsive behaviors that make calling "revealed preference!" on their actions a risky business. The post you linked to (I can't quite tell by your phrasing if you are aware that I wrote it) is about setting priorities between various desiderata, not about declaring some of those desiderata unreal because they take a backseat.

Comment author: Morendil 27 April 2011 07:31:22PM 0 points [-]

aware that I wrote it

Yup.

declaring some of those desiderata unreal because they take a backseat

Not sure if you mean to imply I've been saying that. That wasn't my intention.

Comment author: wedrifid 27 April 2011 05:06:20PM 0 points [-]

By contrast, if I suddenly decide that I want more money, I have a number of avenues by which I could arrange that, at least on a small scale. It would be mistaken of me to conclude from this abundance of available financial opportunity that everyone chooses to have the amount of money they have, and that people with less money are choosing to take fewer of the equally abundant opportunities they share with the rich.

This all seems true with the exception of 'by contrast'. You seem to have clearly illustrated a similarity between weight loss and financial gain. There are things that are under people's control but which things are under a given person's control vary by the individual and the circumstances. In both cases people drastically overestimate the extent to which the outcome is a matter of 'choice'.

Comment author: Alicorn 27 April 2011 05:44:47PM 0 points [-]

The "by contrast" paragraph is meant to illustrate how and why I reject the disanalogy "from both directions".

Comment author: Amanojack 27 April 2011 09:31:39PM *  0 points [-]

The real distinction is between what you want to do now and what you want your future self to do later, though there's some word confusion obscuring that point. English is pretty bad at dealing with these types of distinctions, which is probably why this is a recurring discussion item.

Comment author: Amanojack 27 April 2011 03:26:45PM 1 point [-]

People aren't confused about what they want in any given moment. They want to eat donuts, but they don't want to have eaten donuts. They don't want to exercise, but they do want to have exercised.

Comment author: paper-machine 27 April 2011 03:29:48PM 2 points [-]

This is a pretty good reason humans not as a single moral agent, but as a collection of past, present, and future moral agents.

Comment author: XiXiDu 27 April 2011 04:27:46PM -2 points [-]

Talking about wants isn't necessarily any simpler than talking about shoulds.

Oughts are instrumental and wants are terminal. See my comments here and here.

Comment author: timtyler 27 April 2011 04:32:04PM 4 points [-]

Oughts are instrumental and wants are terminal.

Disagree - I don't think that is supported by the dictionary. For instance, I want more money - which is widely regarded as being instrumental.

Maybe you need to spell out what you actually meant here.

Comment author: XiXiDu 27 April 2011 04:53:28PM *  0 points [-]

For instance, I want more money - which is widely regarded as being instrumental.

Oughts and wants are not mutually exclusive in their first-order desirability. You ought to do what you want is a basic axiom of volition. That implies that you also want what you ought. Yet a distinction, if minor, between ought and want is that the former is often a second-order desire as it is instrumental to the latter primary goal.

Comment author: Peterdjones 27 April 2011 04:54:58PM -1 points [-]

Again, whether you ought to do what you want depends on what you want.

Comment author: NMJablonski 27 April 2011 06:16:51PM *  1 point [-]

Can you demonstrate that what you just said is true?

EDIT: And perhaps provide a definition of "ought"?

Comment author: timtyler 27 April 2011 05:02:07PM *  0 points [-]

Wants are fairly straighforwads, but oughts are often tangled up with society, manipulation and signalling. You appear to be presuming some other definition of ought - without making it terribly clear what it is that you are talking about.

Comment author: XiXiDu 27 April 2011 07:15:44PM -1 points [-]

Wants are fairly straighforwads, but oughts are often tangled up with society, manipulation and signalling.

When it comes to goals then in a sense an intelligent agent is similar to a stone rolling down a hill, both are moving towards a sort of equilibrium. The difference is that intelligence is following more complex trajectories as its ability to read and respond to environmental cues is vastly greater than that of a stone. And that is the reason why we perceive oughts to be mainly a fact about society, you ought not to be indifferent about the goals of other agents if they are instrumental to what you want.

"Ought" statements are subjectively objective as they refer to the interrelationship between your goals and the necessary actions to achieve them. "Ought" statements point out the necessary consistency between means and ends. If you need pursue action X to achieve "want" Y you ought to want to do Y.

Comment author: endoself 26 April 2011 11:58:16PM *  0 points [-]

I agree with most of this. The only reason for using the word morality is when talking to someone who does not realize that "Whatever you want." is the only answer that really can be given to the question of "What should I do next?". (Does that sentence make sense?)

The main thing I have to add to this is what Eliezer describes here. The causal 'reason' that I want people to be happy is because of the desires in my brain, but the motivational 'reason' is because happiness matches {happiness + survival + justice + individuality + ...}, which sounds stupid, but that is how I make decisions; I look for what best matches against that pattern. These two reasons are important to distinguish - "If neutrinos make me believe '2 + 3 = 6', then 2 + 3 = 5". Here, people use that world 'morality' to describe an idealized version of their decision processes rather than to describe the desires embodied in their brain in order to emphasize that point, and also because of the large number of people that find this pseudo-equivalence nonobvious.

Comment author: Peterdjones 27 April 2011 02:14:47PM 1 point [-]

"Whatever you want." is the only answer that really can be given to the question of "What should I do next?".

But that is not the answer if someone wants to murder someone. What you have here is actually a reductio ad absurdam o the simplistic theory that morals=desires.

Comment author: NMJablonski 27 April 2011 02:41:36PM 3 points [-]

It only isn't the answer if you have a problem with that particular person being murdered, or perhaps an objection to killing as a principle. I also would object to wanton, chaotic, and criminal killings, but that is because I have a complex network of preferences that inform that objection, not because murder has some intrinsic property of absolute "wrongness".

It is all preferences, and to think otherwise is the most frequent and absurd delusion still prevalent in rationalist communities. Even when a moralistic rationalist admits that moral truths and absolutes do not exist, they continue operating as if they do. They will say:

"Well, there may not be absolute morality, but we can still tell which actions are best for (survival of human race / equality among humans / etc)."

The survival of the human race is a preference! One which not all possible agents share, as we are all keenly aware of in our discussions of the threat posed by superintelligent AI's that don't share our values. There is no obligation for any mind to adopt any values. You can complain about that reality. You can insist that your preferences are the one, true, good and noble preferences, but no rational agent is obligated, in any empirical sense, to agree with you.

Comment author: Peterdjones 27 April 2011 03:13:10PM *  -1 points [-]

If people have some of the preferences they have because they should have them, the issue of ethics has simply been pushed back a stage. You cannot knock down the whole concept of ethics just by objecting to one simplistic idea, eg. "intrinsic wrongness". Particularly when more complex ideas have been spelt out..

The most frequent and absurd delusion in rationalist circles is that you can arrive at simple solutions to complicated problems by throwing a little science at them.

Rational agents are obliged to believe what can be demonstrated through reasons. Rationality is a norm. Morality is a norm too, if it is anything. You assume tactily that no reasoned demonstration of ethics can be made, but that is just an assumption. You have not done anything like enough to oblige a reasonable person to believe in the elimination of morality.

Comment author: NMJablonski 27 April 2011 05:18:38PM 0 points [-]

Well, when you have something substantive and meaningful to point to let me know. I suggest tabooing words like "ethics", "morality", "should", etc. If you can give me a clear reductionist description of what you're talking about in metaethics without using those words, I'd love to hear it.

Comment author: Peterdjones 27 April 2011 05:30:44PM 1 point [-]

There is no reason I should avoid the words "ethics", "morality", etc, in a discussion of ethics, morality, etc. It is in fact, an unreasonable request on your part.

I am also unpersuaded that I need to be a "reductionist" on the topic. The material on reductionism on this site seems to me a charter for coming up with pseudo-solutions that just sweep the problems under the rug.

My substantive point remains that you have not made a case for eliminating ethics in favour of preferences.

Comment author: torekp 28 April 2011 01:54:28AM 2 points [-]

I'd like to hear more on this charter for pseudo-solutions. What's wrong with the mainstream LW picture? By private message or in this thread (if it's not too tangential) or in a new discussion thread.

Comment author: NMJablonski 27 April 2011 05:51:31PM 2 points [-]

Your substantive point is nonsensical. My physical, real world understanding of intelligent agents includes preferences. It does not include anything presently labeled "morality" and I have no idea what I would apply that label to.

I don't think you have anything concrete down there that you're talking about (I'd be excited to be wrong about this). So you can do your little philosophers dance in a world of poorly anchored words but I'm not going to take you seriously until you start talking about reality.

Comment author: Peterdjones 27 April 2011 06:13:10PM -2 points [-]

If you can't figure out what to apply "morality" to, that is your problem. Most people do not share it.

Comment author: NMJablonski 27 April 2011 06:22:25PM *  3 points [-]

Alright.

I'm going to give this one last shot. Can you explain, succinctly, what you're talking about when you say "morality"?

Comment author: paper-machine 27 April 2011 05:32:33PM 1 point [-]

The material on reductionism on this site seems to me a charter for coming up with pseudo-solutions that just sweep the problems under the rug.

Could you explain this further?

Comment author: Amanojack 27 April 2011 03:02:45PM *  2 points [-]

It just depends on if "should" is interpreted as "what would best fulfill my wants now" or "what would best fulfill your wants now" (or as something else entirely).

We can't make sense of ethical language until we realize different people mean different things by it.

Comment author: Gray 27 April 2011 04:52:17PM 2 points [-]

And that's what morality always was in the first place. It's a way of getting other people to do otherwise than what they wanted to do. No one would be convinced by "I don't want you to kill people", but if you can convince someone that "It is wrong to kill people", then you've created conflict in that person's desires.

I wonder, in the end, if people here truly want to "be rational" about morality. Myself, I'm not rational about morality, I go along with it. I don't critique it in my personal life. For instance, I refuse to murder someone, no matter how rational it might be to murder someone.

Stick to epistemic rationality, and instrumental rationality, but avoid at all costs normative rationality, is my opinion.

Comment author: [deleted] 27 April 2011 05:20:34PM 4 points [-]

And that's what morality always was in the first place. It's a way of getting other people to do otherwise than what they wanted to do. No one would be convinced by "I don't want you to kill people", but if you can convince someone that "It is wrong to kill people", then you've created conflict in that person's desires.

This is a widespread but mistaken theory of morality. After all, we don't - and can't - convincingly say that just any old thing is "wrong". Here, I'll alternate between saying that actually wrong things are wrong, and saying that random things that you don't want are wrong.

Actually wrong: "it's wrong to kill people." Yup, it is. You just don't want it: "it's wrong for you to arrest me just because I stabbed this innocent bystander to death." Yeah, right. Actually wrong: "it's wrong to mug people." No kidding. You just don't want it: "it's wrong for you to lock your door when you leave the house, because it's wrong for you to do anything to prevent me from coming into your house and taking everything you own to sell on the black market". Not convincing.

If there were nothing more to things being wrong than that you use the word "wrong" to get people to do things, then there would be no difference between these four attempts to get people to do something. But there is: in the first and third case, the claim that the action is wrong is true (and therefore makes a convincing argument). In the second and fourth case, the claim is false (and therefore makes for an unconvincing argument).

Sure, you can use the word "wrong" to get people to do things that you want them to do, but you can use a lot of words for that. For example, if you're somebody's mother and you want them to avoid driving when they're very sleepy, you can tell them that it's "dangerous" to drive in that condition. But as with the word "wrong", you can't use the word "dangerous" for just any situation, because it's not true in just any situation. When a proposed action is really dangerous - or really wrong - then you can use that fact to convince them not to pursue that action. But it's still a fact, independent of whether you use it to get other people to do things you want.

Comment author: Amanojack 27 April 2011 08:33:52PM *  0 points [-]

Objective ethics on LW? I'm a little shocked. This whole post is basically argument from popularity (perhaps more accurate to call it argument from convincingness). Judgments of valuation may be universal or quasi-universal, but they are always subjective. Words like "right" and "wrong" (and "innocent" and "own") and other objective moralistic terms obscure this, so let me do some un-obscuring.

If there were nothing more to things being wrong than that you use the word "wrong" to get people to do things, then there would be no difference between these four attempts to get people to do something. But there is: in the first and third case, the claim that the action is wrong is true (and therefore makes a convincing argument). In the second and fourth case, the claim is false (and therefore makes for an unconvincing argument).

You have this backwards: The claim makes a convincing argument (to you and many others), therefore you call the claim "right"; or the claim makes an unconvincing argument against the action, therefore you call the claim "wrong."

Actually wrong: "it's wrong to kill people." Yup, it is. You just don't want it: "it's wrong for you to arrest me just because I stabbed this innocent bystander to death."

Notice you had to tuck in the word "innocent," which already implies your conclusion that it is "actually wrong" to harm the bystander.

Actually wrong: "it's wrong to mug people." No kidding. You just don't want it: "it's wrong for you to lock your door when you leave the house, because it's wrong for you to do anything to prevent me from coming into your house and taking everything you own to sell on the black market".

Here you used the word "own," which again already implies your conclusion that it is wrong to steal it. Both examples are purely circular. Most people are disgusted by killing and theft, and they may be counterproductive from most people's points of view, but that is just about all we can say about the matter - and all we need to say. We are disgusted, so we ban such actions.

Moral right and wrong are not objective facts. The fact that you and I subjectively experience a moral reaction to killing and theft may be an objective fact, but the wrongness itself is not objective, even though it may be universal or near-universal (that is, even though almost everyone else may feel the same way). Universal subjective valuation is not objective valuation (this latter term is, I contend, completely meaningless - unless someone can supply a useful definition).

Although he was speaking in the context of economics, Ludwig von Mises gave the most succinct explanation of why all valuation is subjective when he said, "We originally want or desire an object not because it is agreeable or good, but we call it agreeable or good because we want or desire it."

Comment author: [deleted] 27 April 2011 09:55:24PM 0 points [-]

You have this backwards: The claim makes a convincing argument (to you and many others), therefore you call the claim "right"; or the claim makes an unconvincing argument against the action, therefore you call the claim "wrong."

You could say that about any word in the English language. Let's try this with the word "rain". On many occasions, a person may say "it's raining and therefore you should take an umbrella". On some occasions this claim will be false and people will know that it's false (e.g. because they looked out a window and saw that it wasn't raining), and so the argument will not be convincing.

What you're doing here can be applied to this rain scenario. You could say: 

The claim makes a convincing argument (to you and many others), therefore you call the claim "right"; or the claim makes an unconvincing argument against the action, therefore you call the claim "wrong."

That is, the claim that it's raining makes a convincing argument on some occasions, and on those occasions you call the claim "right". On other occasions, the claim makes an unconvincing argument, and on those occasions you call the claim "wrong".

So there, we've applied your theory about the concept of morality, to the concept of rain. Your theory could equally well be applied to any concept at all. That is, your theory is that when we are convinced by arguments that employ claims about morality, then we call the claims "right". But you could equally well come up with the theory that when we are convinced by arguments that employ claims about rain, then we call the claims "right".

So what have we demonstrated? With your help, we have demonstrated that in this respect, morality is like rain. And like everything else. Morality is like atoms. Morality is like gravity - in this respect. You have highlighted a property of morality which is shared by absolutely everything else in the universe that we have a word for. And this property is, that you can come up with this reverse theory of it, according to which we call claims employing the term "right" when we are convinced by arguments using those claims.

Notice you had to tuck in the word "innocent," which already implies your conclusion that it is "actually wrong" to harm the bystander.

For me to be guilty of begging the question I would have to be trying to prove that a murder was committed in the hypothetical scenario. But it's a hypothetical scenario in which it is specified that the person committed murder.

Here's the hypothetical scenario, more explicit: someone has just committed a murder. He tells a cop, "it would be wrong for you to arrest me". Since it is not, in fact, wrong for the cop to arrest him, then the argument is unconvincing. In this hypothetical scenario, the reason the argument is unconvincing is that it is not actually wrong for the cop to arrest him.

Now, according to your own reverse theory of morality, the hypothetical scenario that I have specified in fact reduces to the following: someone is in a situation where his claims that it would be wrong to arrest him will go ignored by the cop in question. Therefore, the cop believes that it is right to arrest him.

But as I explained before, you can apply your reverse treatment to absolutely anything at all. Here's an example: in this scenario, someone picks up an orange and says about the orange, "this is an apple". Nobody is convinced by his assertion, and the reason nobody is convinced by his assertion is that the orange is, in fact, not an apple.

Now we can apply your reverse treatment to this scenario. Someone picks up something and says about it, "this is an apple". Nobody is convinced by his assertion, and therefore they call his claim "wrong".

Notice the reversal. In my description of the scenario, that the claim is wrong causes others to disbelieve the claim, because they can see with their own eyes that it is wrong. In your reverse description of the scenario, the primary fact is that people are not convinced by the claim, and the secondary fact which follows from the primary fact is that they call the claim "wrong".

You're not proving anything with the reversal, because you can apply the reversal to anything at all.

Here you used the word "own," which again already implies your conclusion that it is wrong to steal it.

Once again, this is a hypothetical scenario in which it is specified that it would be stealing, and therefore wrong. I am not trying to prove that; I am specifying it to construct the scenario.

Although he was speaking in the context of economics, Ludwig von Mises gave the most succinct explanation of why all valuation is subjective when he said, "We originally want or desire an object not because it is agreeable or good, but we call it agreeable or good because we want or desire it."

Absolutely, but morality is not personal preference any more than price is personal preference. These are separate subjects. Mises would not say, "I recognize that the price of gasoline is $4 not because it is $4; rather, the price of gasoline is $4 because I recognize it as $4, and if tomorrow I recognize it as $2 then it will be $2, whatever the gas station attendant says." That would be absurd for him to say. The same applies to morality.

Comment author: Amanojack 27 April 2011 10:25:14PM 0 points [-]

You have this backwards: The claim makes a convincing argument (to you and many others), therefore you call the claim "right"; or the claim makes an unconvincing argument against the action, therefore you call the claim "wrong."

So there, we've applied your theory about the concept of morality, to the concept of rain.

You misread me, though perhaps that was my fault. Does the bold help? I was talking about you (Constant), not "you" in the general sense. I wasn't presenting a theory of morality; I was shedding light on yours by suggesting that you are only calling these things right or wrong because you find the arguments convincing.

Actually wrong: "it's wrong to kill people." Yup, it is. You just don't want it: "it's wrong for you to arrest me just because I stabbed this innocent bystander to death."

Notice you had to tuck in the word "innocent," which already implies your conclusion that it is "actually wrong" to harm the bystander.

For me to be guilty of begging the question I would have to be trying to prove that a murder was committed in the hypothetical scenario.

No, you'd have to be trying to justify your statement that "it is wrong to kill people," which it seems you were (likewise for the theft example). Maybe your unusual phrasing confused me as to what you were trying to show with that. Anyway, the daughter posts seem to show we agree on more than it appears here, so bygones.

As for the rest about "[my] reverse theory of morality," that's all from the above misunderstanding. (Sorry to waste time with my unclear wording.)

Comment author: [deleted] 27 April 2011 11:07:24PM 0 points [-]

You misread me, though perhaps that was my fault. Does the bold help? I was talking about you (Constant), not "you" in the general sense. I wasn't presenting a theory of morality; I was shedding light on yours by suggesting that you are only calling these things right or wrong because you find the arguments convincing.

Okay, but even on this reading you could "shed" similar "light" on absolutely any term that I ever use. You're not proving anything special about morality by that. To do that would require finding differences between morality and, say, rain, or apples. But if we were arguing about apples you could make precisely the same move that you made in this discussion about morality.

Here's a parallel back-and-forth employing apples. Somebody says:

And that's what the concept of apples always was in the first place. It's a way of getting other people to do otherwise than what they wanted to do.

I reply:

[insert examples with apples] In the second and fourth case, the claim about apples is false (and therefore makes for an unconvincing argument).

Here, let me construct an example with apples. Somebody goes to Tiffany's, points to a large diamond on display, and says to an employee, "that is an apple, therefore you should be willing to sell it to me for five dollars, which is a great price for an apple." This claim is false, and therefore makes for an unconvincing argument.

Somebody replies:

You have this backwards: The claim about apples makes a convincing argument (to you and many others), therefore you [Constant] call the claim "right" [true*]; or the claim about apples makes an unconvincing argument against the action, therefore you [Constant] call the claim "wrong" [false*].

* I interpret "right" and "wrong" here as meaning "true" and "false", because claims are true or false, and these are referring to claims here.

To which they follow up:

I wasn't presenting a theory of apples; I was shedding light on your theory of apples by suggesting that you are only calling these things right or wrong [these claims true or false **] because you find the arguments convincing.

** I am continuing the previous interpretation of "right" and "wrong" as meaning, in context here, "true" or "false". If this is not what you meant then I can easily substitute in what you actually meant, make the corresponding changes, and make the same point as I am making here.

What all this boils down to is that my interlocutor is saying that I am only calling claims about apples true or false because I find the arguments that employ these claims convincing or unconvincing. For example, if I happen to be in Tiffany's and somebody points to one of the big shiny glassy-looking things with an enormous price tag and says to an employee, "that is an apple, and therefore you should be happy to accept $5 for it", then I will find that person's argument unconvincing. My interlocutor's point is that I am only calling that person's claim (that that object is an apple) false because I find his argument (that the employee should sell it to him for $5) unconvincing.

Whereas my own account is as follows: I first of all find the person's claim about the shiny glassy thing false. Then, as a consequence, I find his argument (that the employee should be happy to part with it for $5) unconvincing.

If you like I can come up with yet another example, taking place in Tiffany's, dropping the apple, and introducing some action such as grabbing a diamond and attempting to leave the premises. I would have my account (that I, a bystander, saw the man grab the diamond, which I believed to be a wrong act, and therefore when security stopped him I was not persuaded by his claims that he had done nothing wrong), and you would have your reversed account (that I was not persuaded by his claims that he had done nothing wrong, and therefore, as a consequence, I believed his grabbing the diamond to be a wrong act).

Comment author: wedrifid 27 April 2011 05:53:28PM *  -3 points [-]

But as with the word "wrong", you can't use the word "dangerous" for just any situation, because it's not true in just any situation.

Not a good analogy. The objective element of 'wrong' is entirely different in nature to that of 'dangerous' even though by many definitions it does, in fact, exist.

Comment author: [deleted] 27 April 2011 06:31:29PM 4 points [-]

Not a good analogy. The objective element of 'wrong' is entirely different in nature to that of 'dangerous' even though by many definitions it does, in fact, exist.

The word "danger" illustrates a point about logic. The logical point is that the fact that X is often used to persuade people does not mean that the nature of X is that it is " a way of getting other people to do otherwise than what they wanted to do". The common use of the word "danger" is an illustration of this logical point. The illustration is correct.

Comment deleted 27 April 2011 06:43:24PM [-]
Comment author: NMJablonski 27 April 2011 06:46:01PM 3 points [-]

Upvoted to both of you for an interesting discussion. It has reached the point it usually does in metaethics where I have to ask for someone to explain:

What the hell does it mean for something to be objectively wrong?

(This isn't targeted at you specifically wedrifid, it just isn't clear to me what the objectivity of "wrongness" could possibly refer to)

Comment author: [deleted] 27 April 2011 06:52:27PM 1 point [-]

I am not comparing the objectivity of "danger" to the objectivity of "wrong". I am not stating or implying that their objectivity is the same or similar. I am using the word "danger" as an illustration of a point. The point is correct, and the illustration is correct. That "danger" has different objectivity from "wrong" is not relevant to the point I was illustrating.

Comment author: Amanojack 27 April 2011 07:53:30PM *  0 points [-]

And that's what morality always was in the first place. It's a way of getting other people to do otherwise than what they wanted to do. No one would be convinced by "I don't want you to kill people", but if you can convince someone that "It is wrong to kill people", then you've created conflict in that person's desires.

That's one of the things morality has been, and it could indeed be the main thing, but my point above is it all depends on what the person means. Even though getting other people to do something might be the main and most important role of moral language historically, it only invites confusion to overgeneralize here - though I know how tempting it is to simplify all this ethical nonsense floating around in one fell swoop. Some people do simply use "ought" to mean, "It is in your best interest to," without any desire to get the person to do something. Some people mean "God would disapprove," and maybe they really don't care if that makes you refrain from doing it or not, but they're just letting you know. These little counterexamples ruin the generalization, then we're back to square one.

I think the only way to really simplify ethics is to acknowledge that people mean all sorts of things by it, and let each person - if anyone cares - explain what they intended in each case.

No, scratch that. The reason ethics is so confused is precisely because people have tried to simplify a whole bunch of disparate-but-somewhat-interrelated notions into a single type of phrasing. A full explanation of everything that is called "ethics" would require examination of religion, politics, sociology, psychology, and much more.

For most things that we think we want ethics for, such as AI, instead of trying to figure out that complex of sundry notions shoehorned into the category of ethics, I think we'd be better off just assiduously hugging the query for each question we want to answer about how to get the results we want in the "moral" sphere (things that hit on your moral emotions, like empathy, indignation, etc.). Mostly I'm interested in this series of posts for the promise it presents for doing away with most of the confusion generated by wordplay such as "objective ethics," which I consider to be just an artifact of language.

Comment author: CuSithBell 27 April 2011 03:09:01PM 1 point [-]

One thing it seems to be used for around here is "what should you never do even if you should". E.g. it's usually a really bad idea (wrt your own wants) to murder someone, even in a large proportion of cases where you think it's a good idea.

Comment author: Clippy 27 April 2011 03:25:57PM *  0 points [-]

What if "should" is interpreted as "Instantiators of decision theories similar to this one would achieve a higher value on their utility function if similar decision theories would yield this action as output"?

Comment author: endoself 28 April 2011 04:07:18AM -1 points [-]

If you don't want someone to murder, you can try to stop them, but they aren't going to agree to not murder unless they want to.

Comment author: Peterdjones 28 April 2011 01:38:12PM 1 point [-]

Want to before they have had their preferences re arranged by moral exhortation, or after?

Comment author: endoself 30 April 2011 03:52:38AM 0 points [-]

I was referring to only fully logical arguments. Obviously it is possible to prevent someone from murdering by expressing extreme disapproval or locking them up.

Comment author: XiXiDu 27 April 2011 09:00:37AM 0 points [-]

Here, people use that world 'morality' to describe an idealized version of their decision processes...

If you are confused about facts in the world then you are talking about epistemic rationality, why would one invoke 'morality' in this context?

Comment author: endoself 28 April 2011 04:13:31AM -1 points [-]

I'm not sure I understand this. Are you objecting to my use of the word 'idealized', on the grounds that preferences and facts are different things and uncertainty is about facts? I would disagree with that. Someone might have two conflicting but very strong preferences. For example, someone might be opposed to homosexuality based on a feeling of disgust but also have a strong feeling that people should have some sort of right to self-determination. Upon sufficient thought, they may decide that the latter outweighs the former and may stop feeling disgust at homosexuals as a result of that introspection. I believe that this situation is one that occurs regularly among humans.

Comment deleted 27 April 2011 02:52:26AM *  [-]
Comment author: Sniffnoy 27 April 2011 04:42:06AM *  3 points [-]

You correctly point out problems with classical utilitarianism; nonetheless, downvoted for equating utilitarianism in general with classical utilitarianism in particular, as well as being irrelevant to the comment it was replying to. And a few other things.

Comment author: FAWS 26 April 2011 05:06:36PM *  0 points [-]

I agree with you that morality can mostly be framed in terms of volation and an adequate decision theory, but I think you are oversimplifying. For example consider people talking about what other people should want purely for their own good. That might be explainable in terms of projecting their own wants in some way (or perhaps selfish self-delusion), but it doesn't seem like something you could easily predict in advance from reasoning about wants if you were unfamiliar with how people act among each other.

Comment author: timtyler 26 April 2011 08:23:34PM *  -1 points [-]

I am increasingly getting the perception that morality/ethics is useless hogwash.

Am not clear from your comment what your beef is.

The whole talk about morality seems to be nothing more than a signaling game.

No: morality is also to do with how to behave yourself and ways of manipulating others in addition to its signalling role.

Comment author: hairyfigment 29 April 2011 08:57:42PM 0 points [-]

But I have yet to come across a single argument that would warrant the use of any terminology related to moral philosophy.

You just did use it.

Now, in this case we could probably rephrase your statement without too much trouble. But it does not seem at all obvious that doing this for all of our beliefs has positive expected value if we just want to maximize epistemic or instrumental rationality.

Comment author: DanArmak 26 April 2011 10:38:09PM 3 points [-]

Some cognitivists think that [...] Other cognitivists think that [...]

Is there a test of the real world that could tell us that some of them are right and others think wrong? If not, what is the value of describing their thoughts?

It's clear to me that applied and normative ethics deal with real and important questions. They are, respectively, heuristics for certain situations, and analysis of possible failure modes of these heuristics.

But I don't understand what metaethics deals with. You write:

Metaethics: What does moral language mean? Do moral facts exist? If so, what are they like, and are they reducible to natural facts? How can we know whether moral judgments are true or false? Is there a connection between making a moral judgment and being motivated to abide by it? Are moral judgments objective or subjective, relative or absolute? Does it make sense to talk about moral progress?

I don't understand why, given the reduction of these questions to substance, they are nearly as important as the first two categories. In fact, some of these questions seem to me not to reduce to anything interesting. "Does it make sense to talk about moral progress?" seems a question about definitions - given an exact definition of "moral" and "progress", there shouldn't be any empirical fact left to discover in order to answer the question. And the part of the post that discusses the positions of various philosophers gives me a strong feeling of confusion and argument about words.

I expect your next posts will make this clearer, but I wish you had included in this post at least a brief description or example of a question in metaethics that it would be useful to know the answer to. Or, at least, interesting to a reasonably broad audience.

Comment author: Sewing-Machine 27 April 2011 08:16:13PM 4 points [-]

I wish you had included in this post at least a brief description or example of a question in metaethics that it would be useful to know the answer to.

This is nicely put. I second the request: what is a metaethical question that could have a useful answer? It would be especially nice if the usefulness was clear from the question itself, and not from the answer that lukeprog is preparing to give.

Comment author: Peterdjones 27 April 2011 01:29:48PM 0 points [-]

Exact definitions are easy to come by, so long as you are not bothered about correctness. Let morality=42, for instance. If you are bothered about correctness, you need to solve metaethics, the question of what morality is, before you can exactly and correctly define "morality".

I can understand the impatience with philosophy -- "why can't they just solve these problems"--because that was my reaction when I first encountered it some 35 years ago. Did I solve philosophy? I only managed to nibble away at some edges. That's all anyone ever manages.

Comment author: wedrifid 27 April 2011 02:30:26PM 6 points [-]

Let morality=42, for instance.

How dare you! 42 isn't even prime, let alone right.

Comment author: DanArmak 27 April 2011 06:21:03PM 4 points [-]

you need to solve metaethics, the question of what morality is

The problem isn't that I don't know the answer. The problem is that I don't understand the question.

"Morality" is a word. "Understanding morality" is, first of all, understanding what people mean when they use that word. I already know the answer to that question: they mean a complex set of evolved behaviors that have to do with selecting and judging behaviors and other agents. Now that I've answered that question, if you claim there is a further unanswered question, you will need to specify what it is exactly. Otherwise it's no different from saying we must "solve the question of what a Platonic ideal is".

There are many important questions about morality that need to be answered - how exactly people make moral decisions, how to predict and manipulate them, how to modify our own behavior to be more consistent, etc. But these are part of applied and normative ethics. I don't understand what metaethics is.

Comment author: Peterdjones 27 April 2011 07:13:18PM 0 points [-]

Understanding morality is second of all deciding what, if anything, it actually is. Water actually is H2O, but you can use the word without knowing that, and you can't find out what water is just by studying how the word is used.

Comment author: DanArmak 27 April 2011 08:13:12PM *  2 points [-]

I think you don't understand my question.

"Water" is H2O. And we can study H2O.

"Morality" is a complex set of evolved behaviors, etc. We can study those behaviors. This is (ETA:) descriptive ethics. What is metaethics, though?

And do you think there are questions to be asked about morals which are not questions about the different human behaviors that are sometimes labeled as morally relevant? Do you think there exists something in the universe, independent of human beings and the accidents of our evolution, that is called "morals"? The original post indicated that some philosophers think so.

Comment author: Peterdjones 27 April 2011 08:26:39PM 0 points [-]

The study of those behaviours is descriptive ethics. The prescription of those behaviours is normative ethics.

We can ask whether some de facto behaviour we have observed is really moral. And that raises the question of what "really moral" means. And that is metaethics and has a number of possible solutions, positive and negative, which are clearly outlined in the original positing. And metaethics does not vanish just because the Platonic approach is rejected.

Comment author: DanArmak 28 April 2011 07:58:42AM *  2 points [-]

We can also ask whether some de facto behavior is really vorpal. That raises the question of what "really vorpal" means. Luckily, I can tell you what it really means: nothing at all.

If you claim the word "moral" means something that I - and most people who use that word - don't know that it means, then 1) you have to tell us what it means as the start of any discussion instead of asking us what it means, and 2) you should really use a new word for your new idea.

The study of those behaviours is descriptive ethics. The prescription of those behaviours is normative ethics.

Thanks for the correction.

Comment author: Peterdjones 28 April 2011 11:25:25AM 1 point [-]

Luckily, I can tell you what it really means: nothing at all.

Negative solutions are possible, as I said.

If you claim the word "moral" means something that I - and most people who use that word - don't know that it means,

I didn't claim that.. I did say that a precise and correct definition requires coming up with a correct theory. But coming up with a correct theory only requires the imprecise pretheoretical definition, and everyone already has that. (I wasn't asking for it because I don't know it, I was asking for it to remind people that they already have it).

If I had promised a correct theory, I would have implictly promised a post-theoretic definition to go with it. But I didn't make the first promise, so I am not commtted to the second.

The whole thing is aimed as a correction to the ideas that you need to have, or can have, completely clear and accurate deffinitions from the get go.

2) you should really use a new word for your new idea.

People should read carefully, and note that I never claimed to have a New Idea.

Comment author: DanArmak 28 April 2011 02:08:35PM 0 points [-]

Negative solutions are possible, as I said.

I take it you mean negative solutions to the question: does "morality" have a meaning we don't precisely know yet?

What I'm saying is that it's your burden to show that we should be considering this question at all. It's not clear to me what this question means or how and why it arises in your mind.

It's as if you said you were going to spend a year researching exactly what cars mean. And I asked: what does it mean for cars to "mean" something that we don't know? It's clearly not the same as saying the word "cars" refers to something, because it can't refer to something we don't know about; a word is defined only by the way we use it. And cars at least exist as physical objects, unlike morality.

So before we talk about possible answers (or the lack of them), I'm asking you to explain to me the question being discussed. What does the question mean? What kind of objects can be the answer - can morality "mean" that ice cream is sweet, or is that wrong type of answer? What is the test used to judge if an answer is true or false? Is there a possibility two people will never agree even though one of their answers is objectively true (like in literature, and unlike in mathematics)?

The whole thing is aimed as a correction to the ideas that you need to have, or can have, completely clear and accurate deffinitions from the get go.

If we only have an inaccurate definition for morality right now, and someone proposes an accurate one, how can we tell if it's correct?

Comment author: Peterdjones 28 April 2011 02:21:20PM 1 point [-]

No, by negative answers, I mean things like error theories in metaethics.

I think your other questions don't have obvious answers. If you think that the lack of obvious answers should lead to something like "ditch the whole thing", we could have a debate about that. Otherwise, you're not saying anything that hasn't been said already.

Comment author: Perplexed 29 April 2011 05:46:04AM 1 point [-]

Our moral judgments are greatly affected by pointing magnets at the point in our brain that processes theory of mind.

The way this is worded makes it seem that the result is produced by static magnetic fields. And that makes it sound like 19th century pseudo-science.

We use our recently-evolved neocortex to make utilitarian judgments, and deontological judgments tend to come from our older 'chimp' brains.

And the way this is worded makes it seem that you think that the neo-cortex is something that evolved since we separated from the chimps.

Moral naturalists tend to think that moral facts can be accessed simply by doing science.

Do they think that it is important to get the science right? Or is it enough just to signal an interest in pop-science to be recognized as a moral meta-giant?

Comment author: lukeprog 30 April 2011 01:32:44PM *  1 point [-]

the way this is worded makes it seem that you think that the neo-cortex is something that evolved since we separated from the chimps

I was trying to make use of Greene's phrase: 'inner chimp.' But you're right; it's not that accurate. I've adjusted the wording above.

Comment author: Perplexed 30 April 2011 03:16:07PM *  4 points [-]

I was trying to make use of Greene's phrase: 'inner chimp.'

I don't think it is Greene's phrase. I spent some time searching, and can find only one place where he used it - a 2007 RadioLab interview with Krulwich. I would be willing to bet that he was primed to use that phrase by the journalist. He doesn't even use the word chimp in the cited paper.

In any case, Greene's arguments are incoherent even by the usual lax standards of evolutionary psychology and consequentialist naturalistic ethics. He suggests that a consequentialist foundation for ethics is superior to a deontological foundation because 'consequentialist moral intuitions' flow from a more recently evolved portion of the brain.

Now it should be obvious that one cannot jump from 'more recently evolved' to 'superior as a moral basis'. You can't even get from 'more recently evolved' to 'more characteristically human'. Maybe you can get to 'more idiosyncratically human'.

But even that only helps if you are comparing moral judgements on which deontologists and consequentialists differ. But Greene does not do that. Instead of comparing two different judgements about the same situation, he discusses two different situations, in both of which pretty-much everyone's moral intuitions agree. He calls the intuitions that everyone has in one situation 'consequentialist' and the intuitions in the other situation 'deontological'!

Now, most people would object that deontology has nothing to do with intuition. Greene has an answer:

In sum, if it seems that I have simply misunderstood what Kant and deontology are all about, it’s because I am advancing an alternative hypothesis to the standard Kantian/deontological understanding of what Kant and deontology are all about. I am putting forth an empirical hypothesis about the hidden psychological essence of deontology, and it cannot be dismissed a priori for the same reason that tropical islanders cannot know a priori whether ice is a form of water.

And so, having completely restructured the playing field, he reaches the following conclusions:

The argument presented above makes trouble for people in search of rationalist theories that can explain and justify their emotionally-driven deontological moral intuitions. But rationalist deontologists may not be the only ones who should think twice.

The arguments presented above cast doubt on the moral intuitions in question regardless of whether one wishes to justify them in abstract theoretical terms. This is, once again, because these intuitions appear to have been shaped by morally irrelevant factors having to do with the constraints and circumstances of our evolutionary history. This is a problem for anyone who is inclined to stand by these intuitions, and that “anyone” includes nearly everyone.

I’ve referred to these intuitions and the judgments they underwrite as “deontological,” but perhaps it would be more accurate to call them non- consequentialist. After all, you don’t have to be a card-carrying deontologist to think that it’s okay to eat in restaurants when people in the world are starving, that it’s inherently good that criminals suffer for their crimes, and that it would be wrong to push the guy off the footbridge. These judgments are perfectly commonsensical, and it seems that the only people who are inclined to question them are card-carrying consequentialists.

Does that mean that all non-consequentialists need to rethink at least some of their moral commitments? I humbly suggest that the answer is yes.

Let me get this straight. The portions of our brains that generate what Green dubs 'deontological intuitions' are evolutionarily ancient, present in all animals. So Greene dismisses those intuitions as "morally irrelevant" since they ultimately arise from "factors having to do with the constraints and circumstances of our evolutionary history". But our 'consequentialist intuitions' are morally relevant because they come from the neo-cortex; a region of the brain that exists only in mammals and which is particularly enlarged in humans. Yet, somehow, he doesn't think that these intuitions are tainted by the contingent nature of their evolutionary history.

Comment author: AlephNeil 30 April 2011 04:23:25PM 4 points [-]

Now it should be obvious that one cannot jump from 'more recently evolved' to 'superior as a moral basis'.

That's obvious to my prefrontal cortex, but my inner chimp finds the idea desperately appealing.

Comment author: lukeprog 30 April 2011 03:23:52PM 4 points [-]

I remember Greene's position being more nuanced than that, but it's been a while since I read his dissertation. In any case, I'm not defending his view. I only claimed that (in its revised wording) "We use our recently-evolved neocortex to make utilitarian judgments, and deontological judgments tend to come from evolutionarily older parts of our brains."

Comment author: Peterdjones 30 April 2011 04:10:43PM 0 points [-]

That's a distinction that makes sense if deontology is hardwired whilst consequentialism varies with evidence.

Comment author: Garren 27 April 2011 09:20:16AM 1 point [-]

Watching this series with interest. I liked the taboo thing in the first post; reminds me of my favorite Hume quote:

'tis usual for men to use words for ideas, and to talk instead of thinking in their reasonings.

Comment author: Jayson_Virissimo 27 April 2011 01:45:23AM *  1 point [-]

lukeprog, where would you place David Gauthier in your flow chart?

Comment author: ata 25 April 2011 07:16:03PM 1 point [-]

This is not quite correct. The error theorist can hold that a statement like "Murder is not wrong" is true, for they think that murder is not wrong or right.

Should that be "The error theorist can't hold that a statement like 'Murder is not wrong' is true"?

(Also, it's not clear to me that classifying error theory as cognitivist is correct. If it claims that all moral statements are based on a fundamentally mistaken intuition, so that "Murder is wrong" has no more factual content than "Murder is flibberty", then is it not asserting that moral claims are not coherent enough to actually be proper beliefs (even false ones)? (And if classifying a metaethic as cognitivist requires only that it implies that moral claims feel like proper beliefs, not necessarily that they actually are proper beliefs, then that would include emotivism too in most cases.))

Comment author: Alicorn 25 April 2011 07:37:12PM 3 points [-]

This is not quite correct. The error theorist can hold that a statement like "Murder is not wrong" is true, for they think that murder is not wrong or right.

Should that be "The error theorist can't hold that a statement like 'Murder is not wrong' is true"?

No. The error theorist may hold "murder is not wrong" and "murder is not right" to be true. Ey just has to hold "murder is wrong" and "murder is right" to be false, and if ey wants to endorse the "not" statements I guess a rule that "things don't have to be either right or wrong" must operate in the background.

Comment author: lukeprog 25 April 2011 07:21:39PM 0 points [-]

ata,

In this case, I managed to say it correctly the first time. :)

If you're not sure about this stuff, you can read the first chapter of Joyce's 'The Myth of Morality', the central statement of contemporary error theory.

Comment author: ata 25 April 2011 07:37:29PM *  3 points [-]

I can see how an error theorist would agree with "Murder is not wrong" in the same sense in which I'd agree with "Murder is not purple", but it's a strange and not very useful sense. My impression had been that error theorists claim that there are no "right" or "wrong" buckets to sort things into in the first place, rather than proposing that both buckets are there but empty — more like ignosticism than atheism. Am I mistaken about that?

Comment author: Larks 25 April 2011 08:59:43PM *  5 points [-]

Error theorists believe that when people say "Murder is wrong", those people are actually trying to claim that it is a fact that murder has the property of being wrong. However, those people are incorrect (error theorists think) because Murder does not have the property of being wrong - because nothing has the property of being wrong.

It's not about whether or not there are buckets - error theory just says that most people think there is stuff in buckets, but they're incorrect.

Comment author: prase 26 April 2011 08:34:32AM *  4 points [-]

However, those people are wrong (error theorists think) because Murder does not have the property of being wrong - because nothing has the property of being wrong.

I smell a peculiar odour of inconsistency.

(That means, add some modifier, as "morally", to the second "wrong", else it sounds really weird.)

Comment author: lukeprog 25 April 2011 11:33:25PM 0 points [-]

Exactly.

Comment author: grouchymusicologist 25 April 2011 07:05:29PM 1 point [-]

Sometimes one hears the term "moral realism," and in fact that term appears pretty often in your bibliography but not in the main text of your post. Would I be right to think that it comprises everything on the flowchart downstream of the answer "Yes" to the question "Are those beliefs about facts that are constituted by something other than human opinion?"?

Comment author: lukeprog 25 April 2011 07:24:48PM 1 point [-]

There are many definitions of moral realism. See here. But yes, your intuition here is roughly the one about the meaning of 'moral realism' shared by mainstream philosophers.

Comment author: NihilCredo 26 April 2011 10:58:27PM 0 points [-]

How neat is the dichotomy between cognitivists and non-cognitivists? Are there significant philosophical factions holding positions such as

  • "Murder is wrong" is a statement of belief, but it also expresses an emotion (and morality's peculiar charm comes from appealing both to a person's analytical mind and to their instincts)
  • Some people approach morality as a system of beliefs, others as gut reactions, and this is connected to their personalities in interesting ways
  • Or perhaps the same person can shift over time from gut reactions to believing in absolute laws to believing that morality is a natural phenomenon, paralleling for example their growth from infant to child to adult

et cetera?

Comment author: NancyLebovitz 26 April 2011 10:19:03AM 0 points [-]

I'm wondering whether emotive responses lack logical content, and also whether belief-based morality requires emotive backing (failure of utilitarianism--yuck!) to move people to action.

Comment author: Manfred 25 April 2011 07:03:50PM *  0 points [-]

One major debate in moral psychology concerns whether moral judgments require some (defeasible) motivation to adhere to the moral judgment (motivational internalism), or whether one can make a moral judgment without being motivated to adhere to it (motivational externalism).

One of the first two "moral judgements" in this confusing sentence is probably a typo. "Defeasible" just makes things more confusing. Maybe follow the vein of your linked Wikipedia paragraph more closely?

Comment author: thomblake 25 April 2011 10:00:29PM *  0 points [-]

Tangent: I think Ayer's observation was correct but he had the implication backwards. The English sentence "Yuck!" contains the assertion "That is bad." and is truth-apt.

I have launched into arguments with people after they expressed distaste, and I think it was at least properly grammatical. A start: "What's yucky about that?"

Comment author: Yvain 25 April 2011 10:44:58PM 3 points [-]

When I was in Thailand, I saw some local tribesmen eat a popular snack of giant beetles. I said "Yuck!" and couldn't watch them. However, I recognize that there's nothing weirder about eating a bug than about eating a chicken and that they're perfectly healthy and nutritious to people who haven't been raised to fear eating them.

Comment author: Amanojack 27 April 2011 05:30:16PM *  0 points [-]

To interpret "Yuck!" as "That is bad/yucky" is to turn what is ostensibly an expression of subjective experience into an ostensibly "objective" statement. You may as well keep it subjective and interpret it as "I am experiencing revulsion." But you'd have to be a pretty cunning arguer to get into a debate about whether another person is really having a subjective experience of revulsion!

Comment author: thomblake 27 April 2011 05:37:32PM 0 points [-]

It's both - expressing revulsion has a normative component, and so does even experiencing revulsion.

To illustrate: If I eat something and exclaim, "Oishii!", that not only expresses that I am "experiencing deliciousness", but also that the thing I'm tasting "is delicious" - my wife can try it out with the expectation that when she eats it she will also "experience deliciousness". It is a good-tasting thing.

Comment author: Amanojack 27 April 2011 09:44:57PM 3 points [-]

It still sounds just like two people experiencing subjective deliciousness. What if a third person, or a dog, or Clippy, finds it not so delicious?