LESSWRONG
LW

Comment Permalink
Wei Dai50

When I said I'm skeptical that Pluralistic Moral Reductionism is on the right track, I meant that I'm skeptical that it is correct when it take positions like:

But whatever our intended meaning of 'ought' is, the same reasoning applies. Either our intended meaning of 'ought' refers (eventually) to the world of math and physics (in which case the is-ought gap is bridged), or else it doesn't (in which case it fails to refer).

and

It suggests that there is no One True Theory of Morality. (We use moral terms in a variety of ways, and some of those ways refer to different sets of natural facts.)

and that we should proceed to try to solve metaethics on the basis of assuming these are correct. (Perhaps if you believe that these positions are correct, then many metaethical debates are dissolved in your mind, but that's not the case for me, and I think you're probably being too confident if you do consider those debates to be "dissolved".)

And thus, in the spirit of exploring multiple approaches simultaneously, I'm trying to make sure we all have a cursory understanding of the neuroscience of human values.

Ok, I certainly have no objection to that. Except this relatively minor nitpick: since most of the studies are based on non-sentient primates, and the post talked little about what is unique to human values (e.g., influence of culture and deliberative thinking) I think it would be more accurate to refer to it as the neuroscience of primate values.

Reply
XiXiDu-10

...since most of the studies are based on non-sentient primates...

Just came across this, might be relevant / of interest: Similarities Between Macaque and Human Brains

Reply
1lukeprog
Sure. We do know a fair bit about specifically human values, too, but I haven't written much about that yet. I'm curious to know why it is that you disagree with what I'll call Claim 1: ...and what you think is wrong about Claim 2: Do you have the same objections as Vladimir?
See in context
by lukeprog
2 min read
14

22

Recently a friend of mine told me that he and a few others were debating how likely it is that I've 'solved metaethics.' Others on this site have gotten the impression that I'm claiming to have made a fundamental breakthrough that I'm currently keeping a secret, and that's what my metaethics sequence is leading up to. Alas, it isn't the case. The first post in my sequence began:

A few months ago, I predicted that we could solve metaethics in 15 years. To most people, that was outrageously optimistic. But I've updated since then. I think much of metaethics can be solved now (depending on where you draw the boundary around the term 'metaethics'.) My upcoming sequence 'No-Nonsense Metaethics' will solve the part that can be solved, and make headway on the parts of metaethics that aren't yet solved. Solving the easier problems of metaethics will give us a clear and stable platform from which to solve the hard questions of morality.

The part I consider 'solved' is the part discussed in Conceptual Analysis and Moral Theory and Pluralistic Moral Reductionism. These posts represent an application of the lessons learned from Eliezer's free will sequence and his words sequence to the subject of metaethics.

I did this because Eliezer mostly skipped this step in his metaethics sequence, perhaps assuming that readers had already applied these lessons to metaethics to solve the easy problems of metaethics, so he could skip right to discussing the harder problems of metaethics. But I think this move was a source of confusion for many LWers, so I wanted to go back and work through the details of what it looks like to solve the easy parts of metaethics with lessons learned from Eliezer's sequences.

The next part of my metaethics sequence will be devoted to "bringing us all up to speed" on several lines of research that seem relevant to solving open problems in metaethics: the literature on how human values work (in brain and behavior), the literature on extracting preferences from what human brains actually do, and the literature on value extrapolation algorithms. For the most part, these literature sets haven't been discussed on Less Wrong despite their apparent relevance to metaethics, so I'm trying to share them with LW myself (e.g. A Crash Course in the Neuroscience of Human Motivation).

Technically, most of these posts will not be listed as being part of my metaethics sequence, but I will refer to them from posts that are technically part of my metaethics sequence, drawing lessons for metaethics from them.

After "bringing us all up to speed" on these topics and perhaps a couple others, I'll use my metaethics sequence to clarify the open problems in metaethics and suggest some places we can hack away at and perhaps make progress. Thus, my metaethics sequence aims to end with something like a Polymath Project set up for collaboratively solving metaethics problems.

I hope this clarifies my intentions for my metaethics sequence.

Personal Blog

22

Mentioned in
My intentions for my metaethics sequence
New Comment
14 comments, sorted by
top scoring
Click to highlight new comments since:
[-]Wei Dai120

This seems like a good opportunity to put down my overall thoughts on your metaethics sequence so far, in light of this clarification.

I'm skeptical that "Pluralistic Moral Reductionism" is on the right track, or that "A Crash Course in the Neuroscience of Human Motivation" (or more generally, empirical science on primate decision making and valuation mechanisms) is likely to be highly relevant to metaethics. I've been trying to keep this skepticism to myself, in the hope that your main "solution" is yet to come, and that you would get to a detailed explanation of it more quickly if you don't spend time to defend other possibly irrelevant ideas. (I had thought that the main "solution" might be the idea of "empathic metaethics" which you indicated you would explain further in a future post, so that's what I was waiting for.)

For solving a hard problem like metaethics, I think it's a good idea to "explore multiple approaches simultaneously" and "trust your intuitions, but don't waste too much time arguing for them". That's another reason I didn't try harder to talk you out of your current metaethical ideas. But if you're not just taking these ideas as basis for one avenue of exploration, but instead are expecting others to consider them "a clear and stable platform from which to solve the hard questions of morality", that is more problematic. As far as I can tell, such a platform is still nowhere in sight.

Reply
[-]lukeprog40

Allow me to offer further clarifications; I think our intuitions about metaethics may diverge less than you think.

One. Pluralistic Moral Reductionism is not the "right track." In that post I write:

But one must not fall into the trap of thinking that a definition you've stipulated (aloud or in your head) for 'ought' must match up to your intended meaning of 'ought' (to which you don't have introspective access). In fact, I suspect it never does, which is why the conceptual analysis of 'ought' language can go in circles for centuries, and why any stipulated meaning of 'ought' is a fake utility function. To see clearly to our intuitive concept of ought, we'll have to try empathic metaethics...

Plualistic moral reductionism is merely a way of dissolving some common debates in philosophical metaethics, so they don't waste our time and we can move on to tackle the hard problems of metaethics.

Two. I agree that human neuroscience may not turn out to be relevant for metaethics, but it seems plausible that it will turn out to be relevant because (1) perhaps the only value in the universe is the type created when sentient beings value things, and hence we can learn about what has value by studying how sentient beings value things, and (2) if CEV is a good plan for Friendly AI, then we'll be extrapolating human values, and thus it'll be worth knowing what the heck we're extrapolating. And thus, in the spirit of exploring multiple approaches simultaneously, I'm trying to make sure we all have a cursory understanding of the neuroscience of human values.

Three. I don't consider human neuroscience to be "a clear and stable platform from which to solve the hard questions of morality." That phrase was meant to talk about the 'dissolving away' of common metaethical debates (e.g. moral realism vs. antirealism) so that they don't confuse us when we're trying to solve the harder problems of metaethics.

Do you still think we disagree on much? If so, I'm curious to hear what you think we disagree about.

Reply
[-]Wei Dai50

When I said I'm skeptical that Pluralistic Moral Reductionism is on the right track, I meant that I'm skeptical that it is correct when it take positions like:

But whatever our intended meaning of 'ought' is, the same reasoning applies. Either our intended meaning of 'ought' refers (eventually) to the world of math and physics (in which case the is-ought gap is bridged), or else it doesn't (in which case it fails to refer).

and

It suggests that there is no One True Theory of Morality. (We use moral terms in a variety of ways, and some of those ways refer to different sets of natural facts.)

and that we should proceed to try to solve metaethics on the basis of assuming these are correct. (Perhaps if you believe that these positions are correct, then many metaethical debates are dissolved in your mind, but that's not the case for me, and I think you're probably being too confident if you do consider those debates to be "dissolved".)

And thus, in the spirit of exploring multiple approaches simultaneously, I'm trying to make sure we all have a cursory understanding of the neuroscience of human values.

Ok, I certainly have no objection to that. Except this relatively minor nitpick: since most of the studies are based on non-sentient primates, and the post talked little about what is unique to human values (e.g., influence of culture and deliberative thinking) I think it would be more accurate to refer to it as the neuroscience of primate values.

Reply
[-]lukeprog10

I think it would be more accurate to refer to it as the neuroscience of primate values

Sure. We do know a fair bit about specifically human values, too, but I haven't written much about that yet.

I'm curious to know why it is that you disagree with what I'll call Claim 1:

But whatever our intended meaning of 'ought' is, the same reasoning applies. Either our intended meaning of 'ought' refers (eventually) to the world of math and physics (in which case the is-ought gap is bridged), or else it doesn't (in which case it fails to refer).

...and what you think is wrong about Claim 2:

It suggests that there is no One True Theory of Morality. (We use moral terms in a variety of ways, and some of those ways refer to different sets of natural facts.)

Do you have the same objections as Vladimir?

Reply
[-]Wei Dai40

My objections stem mainly from the feeling that when we use moral terms, we may be referring to a shared concept of "normativity", which is also referred to in sentences like:

  • What is the correct decision theory?
  • What is the right prior?
  • What is the right way to handle logical uncertainty?

This may well not be the case, but it is a possibility that I'm not willing to rule out, at least until we better understand what "right" in these sentences mean, and why they are not referring to the same thing as "right" in morality. (Of course there's also the possibility that there are different kinds of normativity that are related in some ways but not identical.)

I'm curious to know why it is that you disagree with what I'll call Claim 1:

But whatever our intended meaning of 'ought' is, the same reasoning applies. Either our intended meaning of 'ought' refers (eventually) to the world of math and physics (in which case the is-ought gap is bridged), or else it doesn't (in which case it fails to refer).

I disagree with your approach of assuming linguistic reductionism. It seems to me that we ought to figure out the intended meanings of each possible word/phrase/sentence, and then conclude that reductionism is true if all language either refers to math and physics, or is clearly meaningless (and we can understand why we thought they had meaning). Assuming reductionism first and then searching for meaning of a word only within math and physics seems to be backwards.

Again, if we're talking about "multiple approaches", I have no objection if you think math and physics are the most promising places to look for meaning of moral terms, but I do not view that as a "clear and stable platform".

...and what you think is wrong about Claim 2:

It suggests that there is no One True Theory of Morality. (We use moral terms in a variety of ways, and some of those ways refer to different sets of natural facts.)

In a previous discussion, you wrote:

I guess one difference between us is that I don't see anything particularly 'wrong' with using stipulative definitions as long as you're aware that they don't match the intended meaning (that we don't have access to yet), whereas you like to characterize stipulative definitions as 'wrong' when they don't match the intended meaning.

So when you say "no One True Theory of Morality" I guess you mean under the stipulative definitions of 'morality'. But when people argue over realism vs anti-realism, they are not arguing over whether people sometimes stipulate different definitions for "morality", but instead are disagreeing over the nature of the intended meaning of 'morality'. When you stipulate "anti-realism" to mean "people sometimes stipulate different definitions for "morality" I think I am justified in calling that "wrong", because you've transformed the question into something that has a clear answer but which nobody is particularly interested in. I don't think you've succeeded in dissolving the question that people are really asking.

Reply
[-]lukeprog10

when we use moral terms, we may be referring to a shared concept of "normativity"... This may well not be the case, but it is a possibility that I'm not willing to rule out...

Agreed.

I disagree with your approach of assuming linguistic reductionism.

Well, but I don't 'assume linguistic reductionism'. What I say is that if the intended meaning of 'ought' refers to structures in math and physics, then linguistic reductionism about normative language is correct, and if it doesn't, then normative language (using its intended meaning) fails to refer (assuming ontological reductionism is true).

But when people argue over realism vs anti-realism, they are not arguing over whether people sometimes stipulate different definitions for "morality", but instead are disagreeing over the nature of the intended meaning of 'morality'.

Philosophers usually are, but not always. One thing I'm trying to avoid here is the 'sneaking in connotations' business performed by, in my example, Bill Craig.

I don't think you've succeeded in dissolving the question that people are really asking.

No, I haven't, and I've tried to be clear about that. But perhaps I need to edit 'Pluralistic Moral Reductionism' with additional clarifications, if it still sounds like I think I've dissolved the question that people are really asking. What I've dissolved is some debates that I see some people engaged in.

Edit: Also, I should add that I'm fairly skeptical of the idea that humans share a concept of morality or normativity. I do intend to write something up on the psychology and neuroscience of mental representations and 'intuitive concepts' to explain why, but I've got several other projects stacked up with priority over that.

Reply
[-]torekp40

What would it mean to share a concept of morality or normativity, or more generally, any concept? If I think of gold as "atomic number 79" and my Aunt Joan thinks of it as "the shiny yellow heavy valuable stuff in certain pieces of jewelry" do we fail to share a concept of gold? If such divergence counts as failure to share the concept, would failure to share concepts of morality be important to metaethics? (On this last question I'm thinking: not so much.)

Reply
[-]lukeprog00

Yeah, I'm not sure exactly what Wei Dai and Vladimir Nesov have in mind when they talk about a shared concept of 'ought' or of 'right'. Will Sawin talks about humans having a cognitive module devoted to the processing of 'ought', which I also find implausible given the last 30 years of psychology and neuroscience. I think I have a different view (than Dai, Nesov, and Sawin) of what concepts are and how they are likely to work, but I'd have to put serious time into a post to explain this clearly, I think. For the moment, those who are interested in the subject should read the SEP articles on concepts and mental representation.

Reply
[-]torekp-20

I disagree with your approach of assuming linguistic reductionism. It seems to me that we ought to figure out the intended meanings of each possible word/phrase/sentence, and then conclude that reductionism is true if all language either refers to math and physics, or is clearly meaningless (and we can understand why we thought they had meaning).

I also have doubts about Luke's linguistic approach, but not on account of reductionism. Reductionism is working well enough elsewhere that it should be the hypothesis of first resort here. In contrast to what you write, I doubt the relevance of intended meanings at all. I prefer an attempt to capture the referents, taking a page from successful scientific reductions.

When Mendel discovered his laws of inheritance, he spoke of heredity "factors". Centuries later Crick and Watson and other scientists made discoveries and hypotheses that amounted, roughly, to the claim that Mendel's factors are sequences of amino acids in DNA molecules. Nobody needed to re-read Mendel's work or examine his cultural context to determine his "original intent". Rather, they posited the equivalence and found that it made good sense of Mendel's factors and the laws in which they were invoked.

Or take ball lightning. The phenomenon is rare and unpredictable, and it may be questionable whether there is really anything to answer to that name. To answer the question of what, if anything, ball lightning is, let's not get various stipulative definitions from various people. Instead, let's try some hypotheses on for size: let's generate some buoyant plasma formations, or some obstructed aerodynamic vortices. Let's see if the vast majority of reports of ball lightning can be explained by one or more of these phenomena. If so, we have discovered what ball lightning is, and linguistic stipulations are beside the point.

Of course, there's no guarantee such an approach can work. But there's no guarantee that stipulative definitions will get us anywhere, either. Stipulation tempts definers to pretend to greater access to their conceptual structures than they actually possess. If they resist that temptation, they will probably resist stipulation too, for lack of use.

Reply
[-]XiXiDu-10

...since most of the studies are based on non-sentient primates...

Just came across this, might be relevant / of interest: Similarities Between Macaque and Human Brains

Reply
[-]Vladimir_Nesov40

I'll comment on the quotes Wei selected (this isn't meant to be related to anything else here, just isolated reaction to Wei's drawing attention to these things):

But whatever our intended meaning of 'ought' is, the same reasoning applies. Either our intended meaning of 'ought' refers (eventually) to the world of math and physics (in which case the is-ought gap is bridged), or else it doesn't (in which case it fails to refer).

It's easy to construct all sorts of interpretations that could be said to be referents of anything else. The question is not well-defined on the level where we talk about "referring" and not including more powerful means of constraining what kinds of "referring" are relevant. Correspondingly, "failing to refer" only makes sense relative to a method of interpretation, and in the case of normative value, discovering correct method of interpretation (relevance-guidance) is more or less the same problem as discovering the referents.

It suggests that there is no One True Theory of Morality. (We use moral terms in a variety of ways, and some of those ways refer to different sets of natural facts.)

We might use moral terms in a variety of ways, but maybe still we should use them in One True Way, in which case there is still One True Theory that describes it.

Reply
[-]Vladimir_Nesov10

If I understand Luke correctly, he includes even under "metaethics" a lot of stuff that is not about normative value, but is merely descriptive about human cognitive behavior.

Reply
[-]lukeprog20

That's correct. 'Metaethics' is a term that typically includes the study of many questions in moral psychology, for example whether or not motivational internalism is true in humans, and how moral judgments happen.

Reply
Moderation Log
14Comments