LESSWRONG
LW

All of Adrià Moret's Comments + Replies

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

I am glad to hear you enjoyed the paper and that our conversation has inspired you to work more on this issue! As I mentioned I now find the worries you lay out in the first paragraph significantly more pressing, thank you for pointing them out!

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y10

I do not think this follows, the "consensus" is that sentience is sufficient for moral status. It is not clearly the case that giving some moral consideration to non-human sentient beings would lead to the scenario you describe. Though see: https://www.tandfonline.com/doi/full/10.1080/21550085.2023.2200724

1RogerDearnaley1y

"Some", or "pro-tanto" unspecified amount of moral consideration, I agree in principle. "Equal" or even "anywhere within a few orders of magnitude of equal", and we go extinct. Ants need ~10,000,000 times less resources per individual than humans, so if you don't give humans around ~10,000,000 times the moral value, we end up extinct in favor of more ants. For even tinier creatures, the ratios are even larger. Explaining why moral weight ought to scale linearly with body weight over many orders of magnitude is a challenging moral position to argue for, but any position that doesn't closely approximate that leads to wildly perverse incentives and the "repugnant conclusion". The most plausible-sounding moral argument I've come up with is that moral weight should be assigned somewhat comparably per-species at a planetary level, and then shared out (equally?) per individual member of a species, so smaller more-numerous species end up with a smaller share per individual. However, given my attitude of ethical system design, I view these sorts of arguments as post-facto political-discussion justifications, and am happy to do what works, and between species of very different sizes, the only thing that works is that moral weight scales roughly linearly with adult body weight (or more accurately, resource needs). I enjoyed Jeff Sebo's paper, thank-you for the reference, and mostly agree with his analysis, if not his moral intuitions — but I really wish he had put some approximate numbers in on occasion to show just how many orders of magnitude the ratios can be between the "large" and "small" things he often discusses. Those words conjure up things within an order of magnitude of each other, not many orders-of-magnitude apart. Words like "vast" and "minute" might have been more appropriate, even before he got on to discussing microbes. But I loved Pascal's Bugging. Overall, thank-you for the inspiration: Due to your paper and this conversation, I'm now working on another po

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y10

These are great points, thank you!

Remember that what the SCEV does is not directly that which the individuals included in it directly want, but what they would want after an extrapolation/reflection process that converged in the most coherent way possible. This means that almost certainly, the result is not the same as if there were no extrapolation process. If there were no extrapolation process, one real possibility is that something like what you suggest, such as sentient dust mites or ants taking over the utility function would indeed occur. But ... (read more)

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y21

What I mean by "moral philosophy literature" is the contemporary moral philosophy literature, I should have been more specific, my bad. And in contemporary philosophy, it is universally accepted (though of course, the might exist one philosopher or another who disagrees) that sentience in the sense understood above as the capacity of having positively or negatively valenced phenomenally conscious experiences is sufficient for moral patienthood. If this is the case, then, it is enough to cite a published work or works in which this is evident. This is why I... (read more)

-1RogerDearnaley1y

OK, note to self: If we manage to create a superintelligece, and give us access to the contemporary moral philosophy literature, it will euthanize us all and feed us to ants. Good to know!

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y20

Thank you! I will for sure read these when I have time. And thank you for your comments!

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y10

Regarding how to take into account the interests of insects and other animals/digital minds see this passage I have to exclude form publication: [SCEV would apply an equal consideration of interests principle] "However, this does not entail that, for instance, if there is a non-negligible chance that dust mites or future large language models are sentient, the strength of their interests should be weighted the same as the strength of the interests of entities that we have good reasons to believe that it is very likely that they are sentient. The degree of ... (read more)

1RogerDearnaley1y

This seems to me like a very key omission. I'm puzzled that you didn't restore it, at least on Less Wrong, even if you had to, for some unexplained reason (involving reviewers, I would assume) omit it from your academic publication. I urge you to do so. However, suppose that, in the near future, biologists established beyond all reasonable doubt that dust mites (for example) did, in fact, sense pain, experience physiological symptoms of distress, and otherwise have senses, and are thus definitely sentient under the standard definition (a relatively simple program of neurological and biochemical experiments, apart from the particularly fine positioning of electrodes required). Once that uncertainty had been eliminated (and doing so is of course a rather urgent matter under your proposed ethical system), would their moral value then deserve equal consideration to that of humans? You say "SCEV would apply an equal consideration of interests principle", so I assume that means yes? Obviously the same limited resources that could support a single human could support many millions of ants. So under your proposed SCEV using equal moral weight, AIs would clearly be strongly morally obligated to drive the human species extinct (as soon as it could do without us, and one would hope humanely). Or, if you added ethical rights for a species as a separate entity, or a prohibition on extinction, drive us down to a minimal safe breeding population. Allowing for genetically-engineered insemination from digital genetic data, that would be a small number of individuals, perhaps O(100), certainly no more then O(1000). (While a human is a good source of vegetarian skin flakes for feeding dust mites, these could more efficiently be vat cultured.) The screed at your link https://www.abolitionist.com/?_gl=1*1iqpkhm*_ga*NzU0NDU1ODY0LjE3MDI5MjUzNDY.*_ga_1MVBX8ZRJ9*MTcwMjkyNTM0NS4xLjEuMTcwMjkyNTUwOS4wLjAuMA.. makes emotionally appealing reading. But it spends only two short paragraphs near

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y10

I am arguing that given that

1. (non-human animals deserve moral consideration, and s-risk are bad (I assume this))

We have reasons to believe 2: (we have some pro-tanto reasons to include them in the process of value learning of an artificial superintelligence instead of only including humans).

There are people (whose objections I address in the paper) that accept 1 but do not accept 2. 1 is not justified for the same reasons as 2. 2 is justified for the reasons I present in the paper. 1 is justified by other arguments about animal ethics and the... (read more)

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y31

Hi Roger, first, the paper is addressed to those who already do believe that all sentient beings deserve moral consideration and that their suffering is morally undesirable. I do not argue for these points in the paper, since they are already universally accepted in the moral philosophy literature.

This is why, for instance, write the following: "sentience in the sense understood above as the capacity of having positively or negatively valenced phenomenally conscious experiences is widely regarded and accepted as a sufficient condition for moral patienthood... (read more)

1RogerDearnaley1y

I think you should state these assumptions more clearly at the beginning of the paper, since you appear to be assuming what you are claiming to prove. You are also making incorrect assumptions about your audience, especially when posting it to Less Wrong. The idea that Coherent Extrapolated Volition, Utilitarianism, or "Human Values" applies only to humans, or perhaps only to sapient beings, is quite widespread on Less Wrong. I'm not deeply familiar with the most recent few decades of the moral philosophy literature, so I won't attempt to argue this in a recent context, if that is what you in fact mean by "the moral philosophy literature" (though I have to say that I do find any claim of the form "absolutely everyone who matters agrees with me" inherently suspicious). However, Philosophy is not a field that has made such rapid recent advances such that one can simply ignore all but the last few decades, and for the moral philosophy literature of the early 20th century and the preceding few millennia (which includes basically every philosopher named in a typical introductory guide to Moral Philosophy), this claim is just blatantly false, even to someone from outside the academic specialty. For example, I am quite certain that Nietzsche, Hobbes, Thomas Aquinas and Plato would all have variously taken issue with the proposition that humans and ants deserve equal moral consideration, if ants can be shown to experience pain (though the Jains would not). Or perhaps you would care to cite quotes from each of them clearly supporting your position? Indeed, for much of the last two millennia, Christian moral philosophy made it entirely clear that they believed animals do not have souls, and thus did not deserve the same moral consideration as humans, and that humans held a unique role in God's plan, as the only creature made in His image and imbued with souls. So claiming that your position is "already universally accepted in the moral philosophy literature" while simply ign

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y10

Yes, and - other points may also be relevant:

(1) Whether there are possible scenarios like these in which the ASI cannot find a way to adequately satisfy all the extrapolated volition of the included beings is not clear. There might not be any such scenarios.

(2) If these scenarios are possible, it is also not clear how likely they are.

(3) There is a subset of s-risks and undesirable outcomes (those coming from cooperation failures between powerful agents) that are a problem to all ambitious value-alignment proposals, including CEV and SCEV.

(4) In part, bec... (read more)

1simon1y

Regarding NicholasKees' point about mob rule vs expansion, I wrote a reply that I moved to another comment. In response to the points in the immediate parent comment: You have to decide, at some point, what you are optimizing for. If you optimize for X, Y will potentially be sacrificed. Some conflicts might be resolvable but ultimately you are making a tradeoff somewhere. And while you haven't taken over yet, other people have a voice as to whether they want to get sacrificed for such a trade-off.

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y31

unlike for other humans, we don't have an instrumental reason to include them in the programmed value calculation, and to precommit to doing so, etc. For animals, it's more of a terminal goal.

First, it seems plausible that, we (in fact) do not have instrumental reason to include all humans. As I argue in section 4.2. There are some humans such as: " children, existing people who've never heard about AI or people with severe physical or cognitive disabilities unable to act on and express their own views on the topic" who, if included, would also only ... (read more)

8simon1y

I note that not everyone considers that implausible, for example Tamsin Leake's QACI takes this view. I disagree with both Tamsin Leake and with you: I think that humans-only, but only humans, makes the most sense. But for concrete reasons, not for free-floating moral reasons. I was writing the following as a response to NicholasKees' comment, but I think it belongs better as a response here: ---------------------------------------- ...imagine you are in a mob in such a "tyranny of the mob" kind of situation, with mob-CEV. For the time being, imagine a small mob. You tell the other mob members: "we should expand the franchise/function to other people not in our mob". OK, should the other mob members agree? * maybe they agree with you that it is right that the function should be expanded to other humans. In which case mob-CEV would do it automatically. * Or they don't agree. And still don't agree after full consideration/extrapolation. If they don't agree, what do you do? Ask Total-Utility-God to strike them down for disobeying the One True Morality? At this point you are stuck, if the mob-CEV AI has made the mob untouchable to entities outside it. But there is something you could have done earlier. Earlier, you could have allied with other humans outside of the mob, to pressure the would-be-mob members to pre-commit to not excluding other humans. And in doing so, you might have insisted on including all humans, not specifically the humans you were explicitly allying with, even if you didn't directly care about everyone, because: * the ally group might shift over time, or people outside the ally group might make their own demands * if the franchise is not set to a solid Schelling point (like all humans) then people currently inside might still worry about the lines being shifted to exclude them. Thus, you include the Sentinelese, not because you're worried about them coming over to demand to be included, but because if you draw the line to exclude

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y10

Okay, I understand better now.

You ask: "Where does your belief regarding the badness of s-risks come from?"

And you provide 3 possible answers I am (in your view) able to choose between:

"From what most people value" 2. "From what I personally value but others don't" or 3. "from pure logic that the rest of us would realize if we were smart enough".

However, the first two answers do not seem to be answers to the question. My beliefs about what is or is not morally desirable do not come from "what most people value" or "what I personally value but o... (read more)

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y1-1

It is not clear to me exactly what "belief regarding suffering" you are talking about, what you mean by "ordinary human values"/"your own personal unique values".

As I argue in Section 2.2., there is (at least) a non-negligible chance that s-risks occur as a result of implementing human-CEV, even if s-risks are very morally undesirable (either in a realist or non-realist sense).

Please read the paper, and if you have any specific points of disagreement cite the passages you would like to discuss. Thank you

0RogerDearnaley1y

Suppose that my definition of "suffering" (as a moral term) was "suffering by a human" and my definition of "s-risk" was "risk of massive risk of suffering by humans", and my definition of 'human' was a member of the biological species Homo sapiens (or a high-fidelity upload of one). You tell me we have to pay attention to animal suffering and animal s-risks, and I say "while the biological phenomenon of pain in humans and in animals is identical, in my ethical system human have moral weight and animals don't. So animal pain is not, morally speaking, suffering, and risk of it is not s-risk." You say "oh yes it is", and I say "by your ethical systems axioms, yes, but not by mine". How do you then persuade me otherwise, using only ethics and logic, when you and I don't operate in the same ethical system? You're just saying "I have axiom A", and my reply is "good for you, I have axiom B". You can't use logic here, because you and your interlocutor don't share the same axiom system. However, you can say "A society that used my proposed ethical system would produce outcome X, whereas a society using yours would produce outcome Y, and pretty-much every human finds X cute and fluffy and Y nauseating, that's just the way human instincts are. So even though all you care about is humans, my ethical system is better." That's a valid argument that might win, ethical logic isn't. You have to appeal to instinct and/or aesthetics, because that's all you and your interlocutor (hopefully) agree on.

1simon1y

Belief regarding suffering: the belief that s-risks are bad, independently of human values as would be represented in CEV. Ordinary human values: what most people have. Your own personal unique values: what you have, but others don't. In my other reply comment, I pointed out disagreements with particular parts of the paper you cited in favour of your views. My fundamental disagreement though, is that you are fundamentally relying on an unjustified assumption, repeated in your comment above: The assumption being that s-risks are "very morally undesirable", independently of human desires (represented in CEV).

Taking Into Account Sentient Non-Humans in AI Ambitious Value Learning: Sentientist Coherent Extrapolated Volition

Adrià Moret1y3-2

Hi simon,

it is not clear to me which of the points of the paper you object to exactly, and I feel some of your worries may already be addressed in the paper.

For instance, you write: "And that's relevant because they are actually existing entities we are working together with on this one planet." First, some sentient non-humans already exist, that is, non-human animals. Second, the fact that we can work or not work with given entities does not seem to be what is relevant in determining whether they should be included in the extrapolation b... (read more)

9simon1y

Thanks for the reply. We don't work together with animals - we act towards them, generously or not. That's key because, unlike for other humans, we don't have an instrumental reason to include them in the programmed value calculation, and to precommit to doing so, etc. For animals, it's more of a terminal goal. But if that terminal goal is a human value, it's represented in CEV. So where does this terminal goal over and above human values come from? Regarding 2: You don't justify why this is a bad thing over and above human values as represented in CEV. Regarding 2.1: You just assume it, that the concept of "moral patients" exists and includes non-humans. Note, to validly claim that CEV is insufficient, it's not enough to say that human values include caring for animals - it has to be something independent of or at least beyond human values. But what? Regarding 4.2: Again, existence and application of the "moral relevance" concept over and above human values just assumed, not justified. regarding 3.2: Good, by focusing at the particular time at least you aren't guaranteeing that the AI will replace us with utility monsters. But if utility monsters do come to exist or be found (e.g. utility monster aliens) for whatever reason, the AI will still side with them, because: Also, I have to remark on: You assert your approach is "the most morally desirable" while disclaiming moral realism. So where does that "most morally desirable" come from? And in response to your comment: The "reasons" are simply unjustified assumptions, like "moral relevance" existing (independent of our values, game theoretic considerations including pre-commitments, etc.) (and yes, you don't explicitly say it exists independent of those things in so many words, but your argument doesn't hold unless they do exist independently).