This is a draft written by FSH, Postdoc at Hong Kong University, as part of the Center for AI Safety Philosophy Fellowship. This draft is meant to solicit feedback.
Abstract:
If a future AI system can enjoy far more well-being than a human per resource, what would be the best way to allocate resources between these future AI and our future descendants? It is obvious that on total utilitarianism, one should give everything to the AI. However, it turns out that every Welfarist axiology on the market also gives this same recommendation. Without resorting to deontological normative theories that suggest that we ought not always to create the world with the most value, or non-welfarist theories that tell us that the world with the most value may not be the world with the most welfare, we propose a new theory that justifies the survival of humanity in the face of overwhelming AI wellbeing. We call this new theory, ``Group Prioritarianism".
Introduction:
In 1974, Robert Nozick famously introduced the idea of the ``utility monster", a being that is so efficient at converting resources into utility, that the best way to maximize utility in the world is to give all resources to this one monster at the cost of all of humanity. Nozick goes on to say the mere possibility of the utility monster refutes ``Utilitarian theories", which imply that one ought to sacrifice all humanity into the maws of this utility monster. Such a consequence is taken to be so outlandish that any theory that suggests that humanity ought to be sacrificed to the monster is, in Nozick's words, ``embarassed" [1974, p.41].
In this paper, we will take as a fixed point that Nozick is right in this case: it really is wrong to sacrifice all humanity to the monster, even if doing so increases total well-being[1]. But even if we take this as a fixed-point, many important questions are left unanswered. For example, should we create these utility monsters if we had the option? And if these monsters do come to exist, and if we ought not to give everything to the monsters, then how much should we give to these monsters? And if Utilitarianism cannot accommodate our intuitions about distributive welfare in this case, then what kind of normative theory can?
These questions have gained a new urgency as the utility monster no longer exists as a mere thought experiment designed for the sole purpose of refuting Utilitarianism. It is possible that in the future, these utility monsters might actually come to exist. Bostrom and Shulman [2020] argue that these utility monsters may indeed be realized as artificial agents, or digital minds, who can be cheap to produce and who can possibly experience hundreds of subjective years of life in a short period of ``wall-clock time". Bostrom and Shulman call these artificial agents, ``super-beneficiaries" (henceforth, ``supers"). This raises questions about whether we should indeed create such supers if we can; and if so, how much of our resources should we give them? To be told that Utilitarianism is false is not enough guidance.
In this paper, I will be developing a novel axiology --- one that is independently motivated by our intuitions regarding fairness and diversity --- to answer these questions and to vindicate the judgment that we should not replace all of humanity with supers. Moreover, our axiology will be welfarist (i.e. it will only take well-being as a value), and it will be aggregationist (i.e. it implies the best world is one that contains the most aggregate value). And finally, we will be assuming consequentialism, which tells us to perform the act that produces the most aggregate value.
Here I want to stress that in making all these assumptions, I am not dismissing other normative theories (like deontology) or other axiologies (like pluralist theories of value) as being obviously false. In fact, I think that these views are not obviously false, and that we may indeed have deontological reasons to not replace humanity[2] for broadly non-consequentialist reasons to prevent things of value from being replaced by other things of even greater value.}. And I think that there may be other values that would be lost if we replaced humanity. However, I think it would be a shame if Team Humanity is open only to deontologists and pluralists. I think all humans should be on Team Humanity and resist the idea that we should be replaced by supers. Unfortunately, the survival of humanity is particularly hard to justify within a welfarist framework when we are faced with the possibility of creating a group of supers. We will go into this challenge in-depth in section I. This paper then this challenge head-on in section II where we motivate and develop our welfarist theory. Finally, in section III, we will discuss objections to the view.
Section I: Resisting Replacement
First, we'll assume that supers require less resources to be sustained compared to a human, and so the same amount of resources used to sustain a human life can be used to sustain even more super lives. Secondly, let's assume we can ``speed up" super-beneficiary digital minds so that they can experience more subjective years of bliss within one objective year of ``wall-clock time". This allows for a super to enjoy more subjective years of well-being than a human ever could. To make things more concrete, let us also assume that for every unit of resources that could be used to sustain a happy human life, we can use that same amount of resources to sustain 10 super lives, each with 100 times more total well-being than the happy human life.
Given these assumptions, consider the following principle:
Beneficent Replacement:
For any population X, if there is a happy human, h, in population X, then a population Y, that is otherwise like X except h is replaced with 10 supers with 100 times more well-being than h, is better than X.
Imagine a future population X1, where all the universe's resources is devoted to supporting 100 humans (a slight underestimate), and imagine a population X100 where instead all the universe's resources are given to ten times as many supers. One can construct a sequence of populations between X1 and X100 such that each member of the sequence has one less human and 10 more supers than the last. Given Beneficent Replacement, each member of the sequence is better than the last, and given the transitivity of the ``better than" relation, the last member of the sequence is the best.
Since Beneficent Replacement implies that a world exclusively of supers is better than any world with a mix of humans and supers, any axiology that does not imply that a world exclusively of supers is the best must deny Beneficent Replacement.
Unfortunately, almost all axiologies imply Beneficent Replacement. It is easy to see, for example, that both totalism and averageism imply Beneficent Replacement since each replacement increases both total and average wellbeing.
Similar things can be said about Prioritarianism. According to Prioritarianism, the overall good of a population is determined by a weighted sum of each individuals wellbeing, where the wellbeing of the worst off ``counts for more" than the wellbeing of the best off. More formally, the overall good of a population is determined by first applying a strictly increasing and concave function on each individual's wellbeing levels and then summing them up. Usually this strictly increasing and concave function is something like the square root function, and so a typical Prioritarian view would say that overall good is determined by:
O(X)=n∑i=1√w(xi)
where O(X) is the overall good of population X, n is the number of individuals in X, and w(xi) is the level of well-being of individual xi in X.
It can easily be seen that even on the Prioritarian view, replacing an individual with a much happier individual will increase the overall good, even if the increase may not be proportional to the increase in total well-being. In other words, replacing everyone with 10 times more people with 100 times more well-being may not make things 1000 times better overall, but it will still make things better overall.
Even ad hoc ``Prioritarian" views that say that there is only value in increasing the wellbeing of an individual up to the point of maximum human level wellbeing, after which there is no value in any more additional wellbeing, would still imply Beneficient Replacement. This is because the sheer fact that you can replace 1 happy human with 10 supers who are at least as happy as the human will always increase overall good.
I take it that Totalism, Averageism, and Prioritarianism are some of the most popular and plausible axiologies. Many more besides these that vindicate Beneficent Replacement will also get you the result that a world exclusively filled with supers is better than any mix of supers and humans. For example, there are critical level utilitarians who argue that there is a threshold of well-being above having barely a life worth living such that anyone who has wellbeing below that threshold contributes negatively to the overall good. These critical level utilitarians would also accept Beneficent Replacement so long as the critical threshold isn't far above the welfare of the best supers (but if that were the case, the critical level utilitarians should advocate for extinction).
Perhaps one can resist Beneficent Replacement by adopting a kind of Person-Affecting View. Person-Affecting views are united by a common emphasis on the value of ``making people happy" over ``making happy people" [Narveson, 1973]. There are two broad categories of Person-Affecting Views. Some views are strong in the sense that they say that a world w′ is worse than a world wonly if there is some person who is worse off in w′ than in w. On a strong view, Beneficient Replacement would be false because there is no individual who is made worse off by replacement because there is no individual in the replacement world who would be worse off[3]
Other views are weak in the sense that they say that a world w′ is worse than a world win one important ``Person-Affecting" dimensiononly if there is a person who is worse off in w′ than in w. However, on this view, a world w′ can still be worse than woverall even if it is not worse off in this Person-Affecting Dimension.
However, adopting either kind of Person-Affecting View is unlikely to help in this context. This is because, firstly, the strong Person-Affecting views are too strong as to be plausible, and even then, they are not even strong enough to say that the world would be worse off without humans. And secondly, the weak Person-Affecting views, although they are more plausible, are unable to resist Beneficent Replacement.
To see the first point, consider the familiar choice about whether to exploit the environment for the benefit of actually and presently existing people (but to the detriment of future people who would actually exist if you chose this option), or whether to conserve the environment for the benefit of a different set of people who would actually exist in the future only if you chose to conserve.
Present Population
Future Population A
Future Population B
Conserve
100
--
100
Exploit
110
50
--
Intuitively, one ought to conserve rather than exploit. However, on strong Person-Affecting views, exploiting is not worse than conserving because no one is harmed if we exploit. Moreover, because people are benefitted if we exploit, then the strong person-affecting theorist should recommend exploitation for the purpose of ``making people happy".
And even if these strong person-affecting views are true, they can at most say that each Beneficent Replacement won't make things better. However, unless there is some reason to think that things get worse with some beneficent replacement, we still do not have an argument that it would be better to have a population with some humans as opposed to one consisting exclusively of supers.
Even less helpful in this context are the weak Person-Affecting views. These views do not have the implausible consequence that the world would be better off if we exploit rather than conserve (they only have the consequence that the world would be better off in one, albeit important, normative dimension). However, weak Person-affecting Views are completely compatible with Beneficent Replacement. They can at most say that each beneficent replacement fails to make the world better in one dimension, but they fail to say that each beneficent replacement won't make the world better overall. So in this context, neither the strong nor weak Person-affecting Views imply that the world is worse off without humanity.
So on a wide variety of views --- totalism, averagism, prioritarianism, sufficientarianism, and strong and weak person-affecting views --- we are left with no reason to think that the world would be worse off if humanity were replaced with supers. And on many of these views, we have strong reason to think that the world would in fact be better if humanity were replaced. If we want to justify future human flourishing, we need to use a different theory. Developing and motivating this theory will be the task of the next section.
Section II: Group Prioritarianism
Welfare matters, but how it matters makes all the difference in how we compare populations. In fact, many of the welfarist views touch upon important considerations that point towards the value of future human flourishing. For example, Prioritarians notice that the amount of good one's well-being contributes to the overall good diminishes as one has more and more well-being. This is a step in the right direction because it implies that benefiting a human (who is usually at a lower level of well-being than a super) by even a bit can matter more than benefiting a super by a lot. But Prioritarians still can't stop Beneficent Replacement because the good of bringing a super from non-existence to good existence will always outweigh the good of bringing any other human into the best of human existence.
At this point, one may have been naturally inclined to think a Person-Affecting View would be helpful. Many Person-Affecting Theorists would say that one does not make the world better by bringing a super from non-existence to good existence. But at the same time, they would have to say that one would not make the world better by bringing a human from non-existence to good existence. So the Person-Affecting Theorist gives no good reason for thinking that it is better to bring a human into existence instead of bringing a super into existence.
I want to bring in the considerations of both the Prioritarian and the Person-Affecting Theorist, but apply just `` one weird trick" to resist Beneficent Replacement. The weakness of both the Prioritarian and the Person-Affecting views are that they do not take into account group welfare as something distinct from the welfare of the individuals of the group. In fact, we do care a lot about group welfare. Consider the following examples:
Job Offer: Two people, A and B, are equally well off as individuals. But A belongs to a historically privileged group and B belongs to a historically marginalized group. Although we can benefit either by the same amount by giving them a job offer, many intuitively think that it would do moregood to benefit the one from the historically marginalized group. One such reason may be that we not only care about fairness between individuals (where one might think that fairness implies that we should seek to benefit the worse off of two individuals), but we also care about fairness between groups (where one might think that fairness implies that we should seek to benefit the worse off of two groups).
Save The Polar Bears:
The Polar bear population is dwindling and you have two choices. One is to set up a nature preserve to save the polar bears. Doing so would allow 100 polar bears to live with 5 units of well-being per year. The other option is to decimate their habitat to build a new mall. The new mall will benefit 1000 humans (whose total and average well-being far exceeds the polar bears') by 1 unit of well-being per year, but at the expense of all polar bear well-being. Even though building the mall would increase total and average well-being, it seems worse to benefit humanity at the cost of eliminating the polar bears.
Reparations:
Two groups of people are now equally well off. However, in the past, one group was oppressed by the other and had significantly less aggregate well-being as a result. Intuitively, it would be better if some of the well-being in the oppressing group were redistributed to the historically oppressed group, even if total well-being remains constant.
In each of these examples, totalist, averagist, and prioritarian welfarist theories will say that it doesn't matter what one does (as in Job Hire and Reparations), and only something like Prioritarianism might say something like saving the Polar Bears is better than making the mall. Examples like these, then, are usually set up as counter-examples to welfarist axiologies. One might point to these example to suggest that there is something other than welfare that matters such that the non-welfare maximizing option may be the better one. For example, perhaps there is intrinsic value to diversity (as in Job Hire), or biodiversity (as in Save the Polar Bears, or in justice (as in Reparations). However, although there really may be these different values apart from welfare, we can vindicate these intuitions without departing from a welfarist axiology. In other words, we can model the value of things like fairness, diversity, and to some extent, justice, within a welfarist axiology. The basic idea is that we can increase total goodness at a faster rate when welfare is distributed to members of less well off groups.
Let us call the view just described Group Prioritarianism. Here is one formal statement of the view that will apply for our purposes. To begin, let X be a population, where a ``population" is represented as a vector <ws(xs),wh(xh)>, where xi is a resource given to a group i (and where xs is the percentage of resources given to the supers and xh the percentage of resources given to humans), wi is a function from a resource xi to the aggregate well-being of group i.[4]Let O be our function on a population, X, to its overall goodness. Here's our definition of Group-Level Prioritarianism
O(X)=g(ws(xs))+g(wh(xh))
where g is our concave ``goodness" function that tells us how much good the wellbeing of a group contributes to the overall good. The fact that our goodness function, g is concave means that the greater a group's aggregate wellbeing, the less that group's marginal well-being would contribute to overall goodness. Supposing that these supers are able to convert resources into well-being 1000 times more efficiently than a human (i.e. ws(x)=1000∗wh(x)), then if there are already many supers enjoying extraordinary levels of well-being such that their contribution to the overall good is high, then it would do moregood to give the same resource to a human even if the total amount of well-being would be less than if it had been given to a super.
Now, if our function g is concave enough, then we can see that there is a point in which a beneficent replacement will make things worse, and not better. This will happen when there are so many supers already enjoying so much aggregate well-being that an additional 10 supers will not matter as much as keeping one additional happy human. In that case, Beneficent Replacement would be false.
In fact, suppose our function g is just the square root function. In this case, we can calculate the point at which things will get worse when we transfer human resources to AI resources by just solving the following constrained optimization problem. Let x be the percentage of the universe's resources that go to humanity and y the percentage of the universe's resources that go to the supers. The important thing is that an AI can produce 1000 times more wellbeing for each resource to support a human, so we will just set wh(x)=x and note that ws(x)=1000∗wh(x).[5] Now the question is to optimize z=√ws(y)+√wh(x)=√1000×y+√x, with the constraint that the percentage of resources that go to the supers and humans add up to 100 (i.e. 100=x+y). The answer is that x≈0.1% and y≈99.9%. So on Group Prioritarianism with a square-root function, we see that the ideal distribution is to give humanity just a tiny slice of the universal pie.[6] Strikingly, if we had instead stipulated that the supers can consume resources ten thousand times more efficiently, then the optimum distribution would be 99.99% to the supers and 0.01% to humanity, just as Bostrom and Schulman [2020] guestimated.
At present, Group Prioritarianism is just a sketch of a family of views that all differ in terms of the concave function g that they adopt, and the number of groups that they admit. First, the concavity of our function g matters. We just happened to use the square root function because of it's simplicity, but we could have also easily used the even more concave natural log function. In that case, the optimal distribution would turn out to be closer to 50/50. Of particular importance, however, is how we individuate groups. If we adopt the view that ``everyone is a special little snowflake" that belongs to their own group, then Group Prioritarianism just becomes Prioritarianism. If, on the other hand, we adopt the view that ``we are all one big happy family" in one group, then Group Prioritarianism just devolves into something like total Utilitarianism or Prioritarianism (depending on how we aggregate the well-being of a group). And as we have seen above, both total utilitarianism and Prioritarianism favors a population exclusively of supers, and so it really matters how we individuate our groups.
So group individuation matters, but for precisely that reason, group individuation is not arbitrary. For example, just because we can gerrymander human populations based on the number of hairs on one's head, or based on some other normatively irrelevant feature, it doesn't mean that any individuation of groups all have equal say on how we ought to redistribute resources. We find it obvious that if we care about groups, we should give more resources to groups that can be individuated based on patterns of historic injustice (for example) as opposed to distributing resources to ensure that people whose last name starts with ``k" get as much as the rest. Though it isn't easy to find the precise theory for how to individuate groups (and it is not the place of this paper to develop such a theory), one seemingly plausible heuristic for any such theory is that the normatively relevant features that individuate groups should be features that help explain vast discrepancies in well-being between groups. For example, if well-being differs drastically along gendered or racial lines, and the different groups experience differing levels of well-being because they are in one group as opposed to another, then it seems that individuating groups based on racial and gendered lines is normatively relevant. In our case, we are considering populations which have at least two groups with vastly different levels of well-being, and the best explanation for why these two groups have different levels of well-being is that one group consists of supers (capable of super-human well-being) and another group does not. So if any group distinctions are normatively relevant, then the distinction between humans and supers is normatively relevant.
By now, it should be obvious how Group Prioritarianism draws inspiration from Prioritarianism. However, I also mentioned at the beginning of this section that we also want to draw insights from the Person-Affecting Views. Just as Person-Affecting Views are characterized by the slogan ``we are in favor of making people happy, but neutral about making happy people", so too do we want Group Prioritarianism to advocate the slogan ``we are in favor of making groups happy, but neutral about making happy groups".
Let us call the principle that populations with different groups are incomparable the ``Group-Affecting Principle". The idea, then, is that one cannot simply make the world better by creating a new group. This is especially implausible if one does nothing to increase the welfare distribution among people, but only change how people are grouped together. For example, if one can compare populations that differ in what groups they contain, then it should be possible increase overall goodness by splitting a monolithic group, G, into two groups Gb and G¬b containing bald people and non-bald people respectively by systematically oppressing people who are bald by just a little bit. In that case, we can imagine that the total goodness of that population will go from g(1000) to g(500−ϵ) for the bald group and g(500) for the non-bald group. Now, recall that for any increasing concave function g, g(x)+g(y)≥g(x+y) (for positive x and y). If ϵ is small enough and g is concave enough, then the resulting split population would contain more goodness than the monolithic population. But this is ridiculous. One cannot make the world better by arbitrarily forming a group via oppression. For reasons like this, we must adopt a ``Group-Affecting Principle".
Moreover, our ``Group-Affecting Principle" is not as obviously implausible as the strong Person-Affecting Principle. Recall that a strong Person-Affecting Principle will tell us to exploit our resources to our benefit now, even if it produces descendants that are much worse off than the descendants we would have if we do not exploit now. This result strikes many as extremely unintuitive, which is why many person-affecting theorists have moved to a weaker view. However, an analogous worry for ``Group Affecting Principles" does not strike me as being too bad. For example, our Group-Affecting Principle will tell us that we should exploit now to our benefit if it produces a new species of insect that have lives worth living, even if conserving now would produce a new species of birds that would have lives better than the insects. To me, I see no reason why we should forfeit our present well-being for the sake of producing a new species of birds as opposed to producing a new species of insects when both the birds and insects would have lives worth living.
Moreover, this ``Group-Affecting Principle" is necessary in our context as well. If we do not have this principle, then we can rerun our argument from Beneficent Replacement by just stipulating that instead of each human being replaced by 10 supers, we can imagine each human being replaced by 10 supers that belong to a new group. If we can compare populations with different groups, it would be hard to resist the conclusion that humanity should be replaced by supers provided the supers are sufficiently diverse. Fortunately, we need not accept that conclusion because that conclusion is inconsistent with Group-Affecting Principles, and Group Affecting Principles are independently plausible.
Hopefully by now, one has a good sense of Group Prioritarianism --- how it is motivated, and how it can resist Beneficent Replacement. The next section discusses some objections to the view.
Section III: Objections to the View
Objection 1: How do we individuate groups?
The viability of this approach depends in large part on how we ought to individuate groups. In the last section, I mentioned that the normatively relevant features that individuate groups should be features that help explain vast discrepancies in well-being between groups. One worry is that this principle can be abused to justify gerry-mandered groups. For example, one might wonder if the group containing people with exactly the same amount of welfare as Bob is a normatively relevant group. Perhaps one can argue that one can explain why anyone in this group has a different level of wellbeing from any other group because people in that group all share the feature of having the same amount of welfare as Bob and no one else. If we can individuate groups in this way, and if no two people have exactly the same amount of welfare, then Group Prioritarianism threatens to just reduce into ordinary Prioritarianism.
To be clear, this objection is not an objection against Group Prioritarianismper se. Rather, this is an objection to a hypothetical theory for group individuation. If we had a theory that said that groups ought to be individuated by virtue of their differences in welfare (no matter how small), then the gerry-mandered group above would count as a counter-example to the theory --- it would be an example of a group that satisfies the principles of the theory without being an example of a normatively relevant group. If that's the case, then we need a new theory of group individuation, not a new axiology that doesn't rely on group individuation.
That being said, I have doubts that we can get a good theory of group individuation. My feeling is that, for whatever set of principles we can concoct, we can give an example of a group that satisfies those principles without being normatively relevant. Does that mean I think that Group Prioritarianism, which essentially relies on the notion of a normatively relevant group, is untenable?
Not at all. We do not need a theory of group individuation for Group Prioritarianism to get off the ground just as we don't a full theory of ``comparative similarity" for David Lewis's theory of counterfactuals to get off the ground. For Lewis, a counterfactual ``if A were the case, then B would be the case" is true just in case B is true in all the closest worlds where A is true, where a world w is closer to actuality than w′ iff w is more similar to actuality than w′. And how does one know whether one world is more similar than another world? We do not do so by referring to a theory of ``comparative similarity" that makes no reference to counterfactuals themselves. All such theories are bound to get things wrong.[7] Rather, our judgments on whether any two worlds are more similar to actuality should be determined by which counterfactuals we think are true. This is not to say that Lewis's theory of counterfactuals is devoid of content. It's true that his theory alone cannot tell us which counterfactuals are true, but that is not the point of the theory (moreover, it would be weird if a theory of the semantics of counterfactuals can settle the question of whether the US would have won WWII if they had not started the Manhattan project). Rather, the point of the theory is to elucidate interesting things about the logic of counterfactuals.
Likewise, Group Prioritarianism is an axiology that essentially relies on the notion of a normatively relevant group in the same way Lewis's theory of counterfactuals relies on the notion of comparative similarity between worlds. Just as one cannot determine comparative similarity without reference to our intuitions about counterfactuals themselves, one cannot determine which groups are normatively relevant without reference to our intuitions about axiology either. For example, in determining whether ``people who have the exact level of welfare as Bob" counts as a normatively relevant group, we must refer to our judgments about group redistribution. For example, we must ask whether it follows that transferring welfare from people who belong to another group to the people who belong to Bob's group makes things better by virtue of improving the welfare of groups? The answer to that question is ``obviously not", for the same reason that a person who claims that they should be given priority for being in ``a minority of one" can only be arguing in bad faith. So for that reason, these gerry-mandered groups are ruled out as normatively relevant.
And just as how Lewis's semantics for counter-factuals can give us useful insights about the logic of counter-factuals without giving us an analysis of ``comparative similarity", so too does our group-based axiology give us useful insights about the logic of group redistributions. For example, Group Prioritarianism tells us that one can better maximize overall welfare by giving priority to groups which consume resources less efficiently. And it tells us that, given a log concave function, and given the welfare levels between humans and supers stipulated above, one ought to split our total resources 50-50 between these two groups. And most crucially, it tells us that Beneficent Replacement is false.
Objection 2: Egyptology?
In Parfit's Reasons and Persons, he gives an influential objection against the averageist view [1984, p.420]. The objection is that, if averageism is true, then the permissibility of continuing the human race, with the certainty that the future will contain many many flourishing individuals, would be considered impermissible if we knew that the very pinnacle of human well-being happened already in Ancient Egypt . The idea is that, even if future generations will have good lives, the addition of future flourishing humans can only bring down the average because nothing can match the heights of Ancient Egyptian welfare. Thus, on the averageist view, it would be impermissible to continue the human race. Moreover, in deciding whether or not we should continue the human race, one must first do some Egyptology to find out exactly how well off Ancient Egyptians really were so as to ensure that we do not lower average human well-being. Of course, this result is ridiculous on two accounts. First, it is implausible that one can make the world worse by bringing into existence flourishing human lives. Secondly, it is implausible that in deciding whether we should continue the human race, we should do some Egyptology. What happened in Egypt stays in Egypt, and it should have no effect on the goodness or badness of continuing human flourishing.
Similarly, our decisions to give certain goods to different people will also be dependent on historical facts on the overall well-being of particular groups. For example, suppose (for simplicity) that the world contained only two groups of people --- Egyptians and non-Egyptians. Suppose Asim and Brandon are equally well off and that Asim is an Egyptian and Brandon a non-Egyptian. If Group Prioritarianism is true, then the relative goodness of giving Asim a chocolate bar over giving giving Brandon a chocolate bar depends on the aggregate well-being of their respective groups. If, for example, the Ancient Egyptians had a golden age and their aggregate well-being exceeded that of Brandon's group, then Group Prioritarianism tells us that we should give the chocolate bar to Brandon. But this is just to say that, in order to know whom to give a chocolate bar, we should do some Egyptology.
This result, however, does not strike me as bad as Parfit's Egyptology objection to Averageism for two reasons. The first is that this objection does not tell us that giving benefits to either group will make things worse, and it certainly doesn't tell us that prolonged existence of any group will make things worse. Secondly, although this example implies that Egyptology is necessary to know which action would produce the most good, I think this sensitivity to historical facts is a feature and not a bug.
In fact, one weakness of standard welfarist axiologies is that they are insensitive to historical facts about group welfare. For example, Prioritarian axiologies that seek to benefit the worst off may cease to recommend that reparations be paid to certain historically disadvantaged and oppressed groups so long as the current welfare of the individuals of that group matches those of their former oppressors. Group Prioritarianism, on the other hand, would say that any additional goodness given to a historically oppressed group doesn't disappear the moment the group ceases to be oppressed. After all, reparations are motivated to compensate for past injustices, not current injustices. So, in deciding how to distribute resources, learning a bit of history, and perhaps some Egyptology, can help.
Objection 3: Implausible Verdicts?
Depending on how we fill out the details of Group Prioritarianism, the view implies some wild conclusions that may undercut its initial motivations. Of course, there are some conclusions it will draw that may be counter-intuitive --- for example, that there will be cases where benefiting a better off individual by a small amount can be better than benefiting a worse-off individual by a large amount, provided that the better off individual is in a much more disadvantaged group. But such an example is just something we have to accept if we want to avoid situations where human well-being is completely crowded out by needy supers who happen to have just below average human well-being. However, the following example is much more worrying for proponents of Group Prioritarianism:
Neglect the Masses: Imagine there are two groups: The Few and The Masses. The Few consists of a handful of fabulously well-off royalty. The Masses consist in an astronomically large population of people whose lives are just barely worth living. You now have a choice on how to spend a rare resource. The resource can either double the well-being of everyone in The Masses, or it can bring into existence a new member of The Few whose life is just okay. Which action would do the most good?
Up till now, we have been assuming that aggregate group welfare is determined by some kind of additive axiology like Totalism or Prioritarianism. Both those views, however, famously imply the Repugnant Conclusion, which states that a population of people whose lives are barely worth living is better than a population of people whose lives are great, provided that the first population is much larger than the second. So, assuming that group welfare is determined by either Totalism or Prioritarianism, we can construct a case where The Masses is a better population than The Few. But in that case, if g is concave enough and the population for The Masses is big enough, Group Prioritarianism implies that we should benefit The Few by bringing into existence an okay life instead of doubling everyone's welfare among The Masses. This result seems anti-thetical to our initial motivation to construct an axiology that benefits disadvantaged groups. So although our view can capture many of our intuitions regarding the value of group welfare, our view will not be able to accommodate them all.
There are two ways to respond to this worry. One way is to find a better way of aggregating group welfare. For example, perhaps aggregate group welfare should be determined by the ``Variable Value" view defended by [Hurka, 1982]. On that view, when the population is low, aggregate value is best increased by increasing the total. But as the population grows, adding more people adds far less value, in which case it would be better to increase the average. Such a view would resist the Repugnant Conclusion and say that there is a point where a population, X, is happy enough and large enough such that there is no other population, Y, with very low average well-being that is better than X by virtue of having a much greater population size.
Indeed, the Variable Value view may find its most natural home in the context of playing the role of the group aggregate function in Group Prioritarianism. Hurka [1982] even begins his paper by hinting at the value of having diversity of groups by quoting Thomas Aquinas: ``Just because an angel is better than a stone, it does not follow that two angels is better than an angel and a stone" [1975, III, 71]. However, the Variable Value view alone cannot resist Beneficent Replacement unless it takes group individuation as being normatively significant, and this is something left out of Hurka's picture. Indeed, if supers and humans are treated as one population, the Variable Value view, by itself, would imply that once we have enough happy supers, we should not even allow for any additional humans to exist, even for free, simply because adding more people brings very little value and lowering the average at this point brings a lot of disvalue. But if we take the Variable Value view as playing the role of being the aggregate group welfare function, we avoid this result.
But Variable Value views have their own problems, and indeed, any view that rejects the Repugnant Conclusion have to pick from a wide array of unattractive features [Arrhenius, 2000]. So another way to respond to our problem is to embrace the result that we should benefit The Few over The Masses in this way. After all, from the perspective of insects, they are the Masses and humanity is The Few. If we concoct a view that systematically benefits The Masses over The Few, then it may be the case that all human resources should go instead to benefiting insects instead.
Then again, any view that always benefits The Few over The Masses would be to our disadvantage once the first supers come into existence. For from their perspective, they are The Few and we are The Masses. Thus, humanity is in an awkward position. Views that seek to increase the population of groups with the highest average well-being would result in humanity being crushed by the overwhelming demands from above (i.e. we should give everything to supers). And views that seek to do everything to increase the welfare of groups with the lowest average well being would result in humanity being crushed by the overwhelming demands from below (i.e. we should give everything to insects). I think the view sketched in this paper is a reasonable middle ground. The view may at times give us unintuitive consequences --- but sacrifices must be made for the sake of human survival.
Conclusion:
If super-beneficiaries ever come, the question about whether the human species should be replaced will be the most important question humanity will have to answer. It is no help that all welfarist axiologies until now advocate for human replacement. For this reason, defenders of humanity may think that there are more goods in this world than just welfare, and that diversity is one of them. In this paper, I accommodate this intuition that diversity is valuable without positing diversity as a value. In the axiology developed here, distributions of resources over diverse groups is valuable because well-being is more valuable when given to less well-off groups.
In many ways, however, the view developed in this paper is under-specified. What is presented here is instead a general axiological framework that is capable of resisting Beneficent Replacement. But in order for this axiology to give more concrete guidance on how to distribute resources, we need to specify (1) what groups there are, and (2) how to aggregate group well-being. These are not easy questions to answer. To answer (1), we need to do more metaphysics and sociology. To answer (2), we need to do even more population ethics. Much work needs to be done to fully flesh out Group Prioritarianism, but it is my hope that the framework presented here may be able to absorb the insights from many disparate areas in the humanities to help settle the question of how to best distribute our resources. But whatever the answers to those two questions may be, I'm confident that one recommendation of our framework will still stand: when the supers come, we should not be replaced.
References
Robert Nozick. Anarchy, State, and Utopia. New York: Basic Books, 1974.
Nick Bostrom and Carl Shulman. Sharing the world with digital minds. 2020.
Jacob M. Nebel. Conservatisms about the valuable. Australasian Journal of Philosophy, 100(1):180–194, 2021. doi: 10.1080/00048402.2020.1861037. Gerald Allan Cohen. Finding Oneself in the Other. Princeton University Press, 2012.
Samuel Scheffler. Death and the Afterlife. New York, NY: Oup Usa, 2013
Samuel Scheffler. Why Worry About Future Generations? Oxford University Press, 2018.
Jan Narveson. Moral problems of population. The Monist, 57(1):62–86, 1973. doi: 10.5840/monist197357134.
Derek Parfit. Reasons and Persons. Oxford, GB: Oxford University Press, 1984.
Thomas Hurka. Value and population size. Ethics, 93(3):496–507, 1982. doi: 10.1086/292462.
Thomas Aquinas. Summa Contra Gentiles. University of Notre Dame Press, 1975.
Gustaf Arrhenius. An impossibility theorem for welfarist axiologies. Economics and Philosophy, 16(2):247–266, 2000. doi: 10.1017/s0266267100000249. 16
Instead of talking about ``utility" we will talk about well-being for two reasons. The first is that the term ``utility" is often used not as units real well-being, but a function that is used to represent the structure of one's preferences. Secondly, whereas ``utility" is often associated with preference-satisfaction, we want to remain neutral about whether well-being is determined by the satisfaction of one's preferences, or whether it is determined by some pleasurable mental state, or even whether it is determined by the possession of a list of objective goods.
See [Nebel, 2021], [Cohen, 2012], and [Scheffler 2018, 2013] for broadly non-consequentialist reasons to prevent things of value from being replaced by other things of even greater value.
Here I am assuming the Incomparability of Non-Existence which states that, for any person x with positive wellbeing in world w, x is neither better off, worse off, nor equally well off in a world w′ where x does not exist in world w.
For now, I am indifferent to how this aggregate well-being is given. It could just be the total well-being of the group, or even the average. My sense, though, is that since the same motivations for Group Prioritarianism stems from ordinary Prioritarianism, the aggregate well-being function, wi, should itself be a Prioritarian weighted sum function. For simplicity, though, we will assume throughout that the aggregate welfare functions just give us the sum of the wellbeing of the individuals in group i.
For example, some attempt to cash out ``comparative similarity" between worlds in terms of overall space-time likeness, but such an analysis, combined with Lewis's theory of counterfactuals, would imply that the counterfactual ``If the president pressed the nuclear launch button during the Korean War, then the button would have malfunctioned" is true simply because a world where a small malfunction happens is overall more like our world than one where there had been a nuclear bomb detonated during the war. Such an attempt to analyze ``comparative similarity" without reference to counterfactuals themselves is misguided. Rather, one should take from this example that the falsity of the above counterfactual tells us that a world where there is a malfunction is not more similar to actuality than a world where there isn't.
This is a draft written by FSH, Postdoc at Hong Kong University, as part of the Center for AI Safety Philosophy Fellowship. This draft is meant to solicit feedback.
Abstract:
If a future AI system can enjoy far more well-being than a human per resource, what would be the best way to allocate resources between these future AI and our future descendants? It is obvious that on total utilitarianism, one should give everything to the AI. However, it turns out that every Welfarist axiology on the market also gives this same recommendation. Without resorting to deontological normative theories that suggest that we ought not always to create the world with the most value, or non-welfarist theories that tell us that the world with the most value may not be the world with the most welfare, we propose a new theory that justifies the survival of humanity in the face of overwhelming AI wellbeing. We call this new theory, ``Group Prioritarianism".
Introduction:
In 1974, Robert Nozick famously introduced the idea of the ``utility monster", a being that is so efficient at converting resources into utility, that the best way to maximize utility in the world is to give all resources to this one monster at the cost of all of humanity. Nozick goes on to say the mere possibility of the utility monster refutes ``Utilitarian theories", which imply that one ought to sacrifice all humanity into the maws of this utility monster. Such a consequence is taken to be so outlandish that any theory that suggests that humanity ought to be sacrificed to the monster is, in Nozick's words, ``embarassed" [1974, p.41].
In this paper, we will take as a fixed point that Nozick is right in this case: it really is wrong to sacrifice all humanity to the monster, even if doing so increases total well-being[1]. But even if we take this as a fixed-point, many important questions are left unanswered. For example, should we create these utility monsters if we had the option? And if these monsters do come to exist, and if we ought not to give everything to the monsters, then how much should we give to these monsters? And if Utilitarianism cannot accommodate our intuitions about distributive welfare in this case, then what kind of normative theory can?
These questions have gained a new urgency as the utility monster no longer exists as a mere thought experiment designed for the sole purpose of refuting Utilitarianism. It is possible that in the future, these utility monsters might actually come to exist. Bostrom and Shulman [2020] argue that these utility monsters may indeed be realized as artificial agents, or digital minds, who can be cheap to produce and who can possibly experience hundreds of subjective years of life in a short period of ``wall-clock time". Bostrom and Shulman call these artificial agents, ``super-beneficiaries" (henceforth, ``supers"). This raises questions about whether we should indeed create such supers if we can; and if so, how much of our resources should we give them? To be told that Utilitarianism is false is not enough guidance.
In this paper, I will be developing a novel axiology --- one that is independently motivated by our intuitions regarding fairness and diversity --- to answer these questions and to vindicate the judgment that we should not replace all of humanity with supers. Moreover, our axiology will be welfarist (i.e. it will only take well-being as a value), and it will be aggregationist (i.e. it implies the best world is one that contains the most aggregate value). And finally, we will be assuming consequentialism, which tells us to perform the act that produces the most aggregate value.
Here I want to stress that in making all these assumptions, I am not dismissing other normative theories (like deontology) or other axiologies (like pluralist theories of value) as being obviously false. In fact, I think that these views are not obviously false, and that we may indeed have deontological reasons to not replace humanity[2] for broadly non-consequentialist reasons to prevent things of value from being replaced by other things of even greater value.}. And I think that there may be other values that would be lost if we replaced humanity. However, I think it would be a shame if Team Humanity is open only to deontologists and pluralists. I think all humans should be on Team Humanity and resist the idea that we should be replaced by supers. Unfortunately, the survival of humanity is particularly hard to justify within a welfarist framework when we are faced with the possibility of creating a group of supers. We will go into this challenge in-depth in section I. This paper then this challenge head-on in section II where we motivate and develop our welfarist theory. Finally, in section III, we will discuss objections to the view.
Section I: Resisting Replacement
First, we'll assume that supers require less resources to be sustained compared to a human, and so the same amount of resources used to sustain a human life can be used to sustain even more super lives. Secondly, let's assume we can ``speed up" super-beneficiary digital minds so that they can experience more subjective years of bliss within one objective year of ``wall-clock time". This allows for a super to enjoy more subjective years of well-being than a human ever could. To make things more concrete, let us also assume that for every unit of resources that could be used to sustain a happy human life, we can use that same amount of resources to sustain 10 super lives, each with 100 times more total well-being than the happy human life.
Given these assumptions, consider the following principle:
Beneficent Replacement:
Imagine a future population X1, where all the universe's resources is devoted to supporting 100 humans (a slight underestimate), and imagine a population X100 where instead all the universe's resources are given to ten times as many supers. One can construct a sequence of populations between X1 and X100 such that each member of the sequence has one less human and 10 more supers than the last. Given Beneficent Replacement, each member of the sequence is better than the last, and given the transitivity of the ``better than" relation, the last member of the sequence is the best.
Since Beneficent Replacement implies that a world exclusively of supers is better than any world with a mix of humans and supers, any axiology that does not imply that a world exclusively of supers is the best must deny Beneficent Replacement.
Unfortunately, almost all axiologies imply Beneficent Replacement. It is easy to see, for example, that both totalism and averageism imply Beneficent Replacement since each replacement increases both total and average wellbeing.
Similar things can be said about Prioritarianism. According to Prioritarianism, the overall good of a population is determined by a weighted sum of each individuals wellbeing, where the wellbeing of the worst off ``counts for more" than the wellbeing of the best off. More formally, the overall good of a population is determined by first applying a strictly increasing and concave function on each individual's wellbeing levels and then summing them up. Usually this strictly increasing and concave function is something like the square root function, and so a typical Prioritarian view would say that overall good is determined by:
O(X)=n∑i=1√w(xi)where O(X) is the overall good of population X, n is the number of individuals in X, and w(xi) is the level of well-being of individual xi in X.
It can easily be seen that even on the Prioritarian view, replacing an individual with a much happier individual will increase the overall good, even if the increase may not be proportional to the increase in total well-being. In other words, replacing everyone with 10 times more people with 100 times more well-being may not make things 1000 times better overall, but it will still make things better overall.
Even ad hoc ``Prioritarian" views that say that there is only value in increasing the wellbeing of an individual up to the point of maximum human level wellbeing, after which there is no value in any more additional wellbeing, would still imply Beneficient Replacement. This is because the sheer fact that you can replace 1 happy human with 10 supers who are at least as happy as the human will always increase overall good.
I take it that Totalism, Averageism, and Prioritarianism are some of the most popular and plausible axiologies. Many more besides these that vindicate Beneficent Replacement will also get you the result that a world exclusively filled with supers is better than any mix of supers and humans. For example, there are critical level utilitarians who argue that there is a threshold of well-being above having barely a life worth living such that anyone who has wellbeing below that threshold contributes negatively to the overall good. These critical level utilitarians would also accept Beneficent Replacement so long as the critical threshold isn't far above the welfare of the best supers (but if that were the case, the critical level utilitarians should advocate for extinction).
Perhaps one can resist Beneficent Replacement by adopting a kind of Person-Affecting View. Person-Affecting views are united by a common emphasis on the value of ``making people happy" over ``making happy people" [Narveson, 1973]. There are two broad categories of Person-Affecting Views. Some views are strong in the sense that they say that a world w′ is worse than a world w only if there is some person who is worse off in w′ than in w. On a strong view, Beneficient Replacement would be false because there is no individual who is made worse off by replacement because there is no individual in the replacement world who would be worse off[3]
Other views are weak in the sense that they say that a world w′ is worse than a world w in one important ``Person-Affecting" dimension only if there is a person who is worse off in w′ than in w. However, on this view, a world w′ can still be worse than w overall even if it is not worse off in this Person-Affecting Dimension.
However, adopting either kind of Person-Affecting View is unlikely to help in this context. This is because, firstly, the strong Person-Affecting views are too strong as to be plausible, and even then, they are not even strong enough to say that the world would be worse off without humans. And secondly, the weak Person-Affecting views, although they are more plausible, are unable to resist Beneficent Replacement.
To see the first point, consider the familiar choice about whether to exploit the environment for the benefit of actually and presently existing people (but to the detriment of future people who would actually exist if you chose this option), or whether to conserve the environment for the benefit of a different set of people who would actually exist in the future only if you chose to conserve.
Intuitively, one ought to conserve rather than exploit. However, on strong Person-Affecting views, exploiting is not worse than conserving because no one is harmed if we exploit. Moreover, because people are benefitted if we exploit, then the strong person-affecting theorist should recommend exploitation for the purpose of ``making people happy".
And even if these strong person-affecting views are true, they can at most say that each Beneficent Replacement won't make things better. However, unless there is some reason to think that things get worse with some beneficent replacement, we still do not have an argument that it would be better to have a population with some humans as opposed to one consisting exclusively of supers.
Even less helpful in this context are the weak Person-Affecting views. These views do not have the implausible consequence that the world would be better off if we exploit rather than conserve (they only have the consequence that the world would be better off in one, albeit important, normative dimension). However, weak Person-affecting Views are completely compatible with Beneficent Replacement. They can at most say that each beneficent replacement fails to make the world better in one dimension, but they fail to say that each beneficent replacement won't make the world better overall. So in this context, neither the strong nor weak Person-affecting Views imply that the world is worse off without humanity.
So on a wide variety of views --- totalism, averagism, prioritarianism, sufficientarianism, and strong and weak person-affecting views --- we are left with no reason to think that the world would be worse off if humanity were replaced with supers. And on many of these views, we have strong reason to think that the world would in fact be better if humanity were replaced. If we want to justify future human flourishing, we need to use a different theory. Developing and motivating this theory will be the task of the next section.
Section II: Group Prioritarianism
Welfare matters, but how it matters makes all the difference in how we compare populations. In fact, many of the welfarist views touch upon important considerations that point towards the value of future human flourishing. For example, Prioritarians notice that the amount of good one's well-being contributes to the overall good diminishes as one has more and more well-being. This is a step in the right direction because it implies that benefiting a human (who is usually at a lower level of well-being than a super) by even a bit can matter more than benefiting a super by a lot. But Prioritarians still can't stop Beneficent Replacement because the good of bringing a super from non-existence to good existence will always outweigh the good of bringing any other human into the best of human existence.
At this point, one may have been naturally inclined to think a Person-Affecting View would be helpful. Many Person-Affecting Theorists would say that one does not make the world better by bringing a super from non-existence to good existence. But at the same time, they would have to say that one would not make the world better by bringing a human from non-existence to good existence. So the Person-Affecting Theorist gives no good reason for thinking that it is better to bring a human into existence instead of bringing a super into existence.
I want to bring in the considerations of both the Prioritarian and the Person-Affecting Theorist, but apply just `` one weird trick" to resist Beneficent Replacement. The weakness of both the Prioritarian and the Person-Affecting views are that they do not take into account group welfare as something distinct from the welfare of the individuals of the group. In fact, we do care a lot about group welfare. Consider the following examples:
In each of these examples, totalist, averagist, and prioritarian welfarist theories will say that it doesn't matter what one does (as in Job Hire and Reparations), and only something like Prioritarianism might say something like saving the Polar Bears is better than making the mall. Examples like these, then, are usually set up as counter-examples to welfarist axiologies. One might point to these example to suggest that there is something other than welfare that matters such that the non-welfare maximizing option may be the better one. For example, perhaps there is intrinsic value to diversity (as in Job Hire), or biodiversity (as in Save the Polar Bears, or in justice (as in Reparations). However, although there really may be these different values apart from welfare, we can vindicate these intuitions without departing from a welfarist axiology. In other words, we can model the value of things like fairness, diversity, and to some extent, justice, within a welfarist axiology. The basic idea is that we can increase total goodness at a faster rate when welfare is distributed to members of less well off groups.
Let us call the view just described Group Prioritarianism. Here is one formal statement of the view that will apply for our purposes. To begin, let X be a population, where a ``population" is represented as a vector <ws(xs),wh(xh)>, where xi is a resource given to a group i (and where xs is the percentage of resources given to the supers and xh the percentage of resources given to humans), wi is a function from a resource xi to the aggregate well-being of group i.[4]Let O be our function on a population, X, to its overall goodness. Here's our definition of Group-Level Prioritarianism
O(X)=g(ws(xs))+g(wh(xh))
where g is our concave ``goodness" function that tells us how much good the wellbeing of a group contributes to the overall good. The fact that our goodness function, g is concave means that the greater a group's aggregate wellbeing, the less that group's marginal well-being would contribute to overall goodness. Supposing that these supers are able to convert resources into well-being 1000 times more efficiently than a human (i.e. ws(x)=1000∗wh(x)), then if there are already many supers enjoying extraordinary levels of well-being such that their contribution to the overall good is high, then it would do more good to give the same resource to a human even if the total amount of well-being would be less than if it had been given to a super.
Now, if our function g is concave enough, then we can see that there is a point in which a beneficent replacement will make things worse, and not better. This will happen when there are so many supers already enjoying so much aggregate well-being that an additional 10 supers will not matter as much as keeping one additional happy human. In that case, Beneficent Replacement would be false.
In fact, suppose our function g is just the square root function. In this case, we can calculate the point at which things will get worse when we transfer human resources to AI resources by just solving the following constrained optimization problem. Let x be the percentage of the universe's resources that go to humanity and y the percentage of the universe's resources that go to the supers. The important thing is that an AI can produce 1000 times more wellbeing for each resource to support a human, so we will just set wh(x)=x and note that ws(x)=1000∗wh(x).[5] Now the question is to optimize z=√ws(y)+√wh(x)=√1000×y+√x, with the constraint that the percentage of resources that go to the supers and humans add up to 100 (i.e. 100=x+y). The answer is that x≈0.1% and y≈99.9%. So on Group Prioritarianism with a square-root function, we see that the ideal distribution is to give humanity just a tiny slice of the universal pie.[6] Strikingly, if we had instead stipulated that the supers can consume resources ten thousand times more efficiently, then the optimum distribution would be 99.99% to the supers and 0.01% to humanity, just as Bostrom and Schulman [2020] guestimated.
At present, Group Prioritarianism is just a sketch of a family of views that all differ in terms of the concave function g that they adopt, and the number of groups that they admit. First, the concavity of our function g matters. We just happened to use the square root function because of it's simplicity, but we could have also easily used the even more concave natural log function. In that case, the optimal distribution would turn out to be closer to 50/50. Of particular importance, however, is how we individuate groups. If we adopt the view that ``everyone is a special little snowflake" that belongs to their own group, then Group Prioritarianism just becomes Prioritarianism. If, on the other hand, we adopt the view that ``we are all one big happy family" in one group, then Group Prioritarianism just devolves into something like total Utilitarianism or Prioritarianism (depending on how we aggregate the well-being of a group). And as we have seen above, both total utilitarianism and Prioritarianism favors a population exclusively of supers, and so it really matters how we individuate our groups.
So group individuation matters, but for precisely that reason, group individuation is not arbitrary. For example, just because we can gerrymander human populations based on the number of hairs on one's head, or based on some other normatively irrelevant feature, it doesn't mean that any individuation of groups all have equal say on how we ought to redistribute resources. We find it obvious that if we care about groups, we should give more resources to groups that can be individuated based on patterns of historic injustice (for example) as opposed to distributing resources to ensure that people whose last name starts with ``k" get as much as the rest. Though it isn't easy to find the precise theory for how to individuate groups (and it is not the place of this paper to develop such a theory), one seemingly plausible heuristic for any such theory is that the normatively relevant features that individuate groups should be features that help explain vast discrepancies in well-being between groups. For example, if well-being differs drastically along gendered or racial lines, and the different groups experience differing levels of well-being because they are in one group as opposed to another, then it seems that individuating groups based on racial and gendered lines is normatively relevant. In our case, we are considering populations which have at least two groups with vastly different levels of well-being, and the best explanation for why these two groups have different levels of well-being is that one group consists of supers (capable of super-human well-being) and another group does not. So if any group distinctions are normatively relevant, then the distinction between humans and supers is normatively relevant.
By now, it should be obvious how Group Prioritarianism draws inspiration from Prioritarianism. However, I also mentioned at the beginning of this section that we also want to draw insights from the Person-Affecting Views. Just as Person-Affecting Views are characterized by the slogan ``we are in favor of making people happy, but neutral about making happy people", so too do we want Group Prioritarianism to advocate the slogan ``we are in favor of making groups happy, but neutral about making happy groups".
Let us call the principle that populations with different groups are incomparable the ``Group-Affecting Principle". The idea, then, is that one cannot simply make the world better by creating a new group. This is especially implausible if one does nothing to increase the welfare distribution among people, but only change how people are grouped together. For example, if one can compare populations that differ in what groups they contain, then it should be possible increase overall goodness by splitting a monolithic group, G, into two groups Gb and G¬b containing bald people and non-bald people respectively by systematically oppressing people who are bald by just a little bit. In that case, we can imagine that the total goodness of that population will go from g(1000) to g(500−ϵ) for the bald group and g(500) for the non-bald group. Now, recall that for any increasing concave function g, g(x)+g(y)≥g(x+y) (for positive x and y). If ϵ is small enough and g is concave enough, then the resulting split population would contain more goodness than the monolithic population. But this is ridiculous. One cannot make the world better by arbitrarily forming a group via oppression. For reasons like this, we must adopt a ``Group-Affecting Principle".
Moreover, our ``Group-Affecting Principle" is not as obviously implausible as the strong Person-Affecting Principle. Recall that a strong Person-Affecting Principle will tell us to exploit our resources to our benefit now, even if it produces descendants that are much worse off than the descendants we would have if we do not exploit now. This result strikes many as extremely unintuitive, which is why many person-affecting theorists have moved to a weaker view. However, an analogous worry for ``Group Affecting Principles" does not strike me as being too bad. For example, our Group-Affecting Principle will tell us that we should exploit now to our benefit if it produces a new species of insect that have lives worth living, even if conserving now would produce a new species of birds that would have lives better than the insects. To me, I see no reason why we should forfeit our present well-being for the sake of producing a new species of birds as opposed to producing a new species of insects when both the birds and insects would have lives worth living.
Moreover, this ``Group-Affecting Principle" is necessary in our context as well. If we do not have this principle, then we can rerun our argument from Beneficent Replacement by just stipulating that instead of each human being replaced by 10 supers, we can imagine each human being replaced by 10 supers that belong to a new group. If we can compare populations with different groups, it would be hard to resist the conclusion that humanity should be replaced by supers provided the supers are sufficiently diverse. Fortunately, we need not accept that conclusion because that conclusion is inconsistent with Group-Affecting Principles, and Group Affecting Principles are independently plausible.
Hopefully by now, one has a good sense of Group Prioritarianism --- how it is motivated, and how it can resist Beneficent Replacement. The next section discusses some objections to the view.
Section III: Objections to the View
Objection 1: How do we individuate groups?
The viability of this approach depends in large part on how we ought to individuate groups. In the last section, I mentioned that the normatively relevant features that individuate groups should be features that help explain vast discrepancies in well-being between groups. One worry is that this principle can be abused to justify gerry-mandered groups. For example, one might wonder if the group containing people with exactly the same amount of welfare as Bob is a normatively relevant group. Perhaps one can argue that one can explain why anyone in this group has a different level of wellbeing from any other group because people in that group all share the feature of having the same amount of welfare as Bob and no one else. If we can individuate groups in this way, and if no two people have exactly the same amount of welfare, then Group Prioritarianism threatens to just reduce into ordinary Prioritarianism.
To be clear, this objection is not an objection against Group Prioritarianism per se. Rather, this is an objection to a hypothetical theory for group individuation. If we had a theory that said that groups ought to be individuated by virtue of their differences in welfare (no matter how small), then the gerry-mandered group above would count as a counter-example to the theory --- it would be an example of a group that satisfies the principles of the theory without being an example of a normatively relevant group. If that's the case, then we need a new theory of group individuation, not a new axiology that doesn't rely on group individuation.
That being said, I have doubts that we can get a good theory of group individuation. My feeling is that, for whatever set of principles we can concoct, we can give an example of a group that satisfies those principles without being normatively relevant. Does that mean I think that Group Prioritarianism, which essentially relies on the notion of a normatively relevant group, is untenable?
Not at all. We do not need a theory of group individuation for Group Prioritarianism to get off the ground just as we don't a full theory of ``comparative similarity" for David Lewis's theory of counterfactuals to get off the ground. For Lewis, a counterfactual ``if A were the case, then B would be the case" is true just in case B is true in all the closest worlds where A is true, where a world w is closer to actuality than w′ iff w is more similar to actuality than w′. And how does one know whether one world is more similar than another world? We do not do so by referring to a theory of ``comparative similarity" that makes no reference to counterfactuals themselves. All such theories are bound to get things wrong.[7] Rather, our judgments on whether any two worlds are more similar to actuality should be determined by which counterfactuals we think are true. This is not to say that Lewis's theory of counterfactuals is devoid of content. It's true that his theory alone cannot tell us which counterfactuals are true, but that is not the point of the theory (moreover, it would be weird if a theory of the semantics of counterfactuals can settle the question of whether the US would have won WWII if they had not started the Manhattan project). Rather, the point of the theory is to elucidate interesting things about the logic of counterfactuals.
Likewise, Group Prioritarianism is an axiology that essentially relies on the notion of a normatively relevant group in the same way Lewis's theory of counterfactuals relies on the notion of comparative similarity between worlds. Just as one cannot determine comparative similarity without reference to our intuitions about counterfactuals themselves, one cannot determine which groups are normatively relevant without reference to our intuitions about axiology either. For example, in determining whether ``people who have the exact level of welfare as Bob" counts as a normatively relevant group, we must refer to our judgments about group redistribution. For example, we must ask whether it follows that transferring welfare from people who belong to another group to the people who belong to Bob's group makes things better by virtue of improving the welfare of groups? The answer to that question is ``obviously not", for the same reason that a person who claims that they should be given priority for being in ``a minority of one" can only be arguing in bad faith. So for that reason, these gerry-mandered groups are ruled out as normatively relevant.
And just as how Lewis's semantics for counter-factuals can give us useful insights about the logic of counter-factuals without giving us an analysis of ``comparative similarity", so too does our group-based axiology give us useful insights about the logic of group redistributions. For example, Group Prioritarianism tells us that one can better maximize overall welfare by giving priority to groups which consume resources less efficiently. And it tells us that, given a log concave function, and given the welfare levels between humans and supers stipulated above, one ought to split our total resources 50-50 between these two groups. And most crucially, it tells us that Beneficent Replacement is false.
Objection 2: Egyptology?
In Parfit's Reasons and Persons, he gives an influential objection against the averageist view [1984, p.420]. The objection is that, if averageism is true, then the permissibility of continuing the human race, with the certainty that the future will contain many many flourishing individuals, would be considered impermissible if we knew that the very pinnacle of human well-being happened already in Ancient Egypt . The idea is that, even if future generations will have good lives, the addition of future flourishing humans can only bring down the average because nothing can match the heights of Ancient Egyptian welfare. Thus, on the averageist view, it would be impermissible to continue the human race. Moreover, in deciding whether or not we should continue the human race, one must first do some Egyptology to find out exactly how well off Ancient Egyptians really were so as to ensure that we do not lower average human well-being. Of course, this result is ridiculous on two accounts. First, it is implausible that one can make the world worse by bringing into existence flourishing human lives. Secondly, it is implausible that in deciding whether we should continue the human race, we should do some Egyptology. What happened in Egypt stays in Egypt, and it should have no effect on the goodness or badness of continuing human flourishing.
Similarly, our decisions to give certain goods to different people will also be dependent on historical facts on the overall well-being of particular groups. For example, suppose (for simplicity) that the world contained only two groups of people --- Egyptians and non-Egyptians. Suppose Asim and Brandon are equally well off and that Asim is an Egyptian and Brandon a non-Egyptian. If Group Prioritarianism is true, then the relative goodness of giving Asim a chocolate bar over giving giving Brandon a chocolate bar depends on the aggregate well-being of their respective groups. If, for example, the Ancient Egyptians had a golden age and their aggregate well-being exceeded that of Brandon's group, then Group Prioritarianism tells us that we should give the chocolate bar to Brandon. But this is just to say that, in order to know whom to give a chocolate bar, we should do some Egyptology.
This result, however, does not strike me as bad as Parfit's Egyptology objection to Averageism for two reasons. The first is that this objection does not tell us that giving benefits to either group will make things worse, and it certainly doesn't tell us that prolonged existence of any group will make things worse. Secondly, although this example implies that Egyptology is necessary to know which action would produce the most good, I think this sensitivity to historical facts is a feature and not a bug.
In fact, one weakness of standard welfarist axiologies is that they are insensitive to historical facts about group welfare. For example, Prioritarian axiologies that seek to benefit the worst off may cease to recommend that reparations be paid to certain historically disadvantaged and oppressed groups so long as the current welfare of the individuals of that group matches those of their former oppressors. Group Prioritarianism, on the other hand, would say that any additional goodness given to a historically oppressed group doesn't disappear the moment the group ceases to be oppressed. After all, reparations are motivated to compensate for past injustices, not current injustices. So, in deciding how to distribute resources, learning a bit of history, and perhaps some Egyptology, can help.
Objection 3: Implausible Verdicts?
Depending on how we fill out the details of Group Prioritarianism, the view implies some wild conclusions that may undercut its initial motivations. Of course, there are some conclusions it will draw that may be counter-intuitive --- for example, that there will be cases where benefiting a better off individual by a small amount can be better than benefiting a worse-off individual by a large amount, provided that the better off individual is in a much more disadvantaged group. But such an example is just something we have to accept if we want to avoid situations where human well-being is completely crowded out by needy supers who happen to have just below average human well-being. However, the following example is much more worrying for proponents of Group Prioritarianism:
Up till now, we have been assuming that aggregate group welfare is determined by some kind of additive axiology like Totalism or Prioritarianism. Both those views, however, famously imply the Repugnant Conclusion, which states that a population of people whose lives are barely worth living is better than a population of people whose lives are great, provided that the first population is much larger than the second. So, assuming that group welfare is determined by either Totalism or Prioritarianism, we can construct a case where The Masses is a better population than The Few. But in that case, if g is concave enough and the population for The Masses is big enough, Group Prioritarianism implies that we should benefit The Few by bringing into existence an okay life instead of doubling everyone's welfare among The Masses. This result seems anti-thetical to our initial motivation to construct an axiology that benefits disadvantaged groups. So although our view can capture many of our intuitions regarding the value of group welfare, our view will not be able to accommodate them all.
There are two ways to respond to this worry. One way is to find a better way of aggregating group welfare. For example, perhaps aggregate group welfare should be determined by the ``Variable Value" view defended by [Hurka, 1982]. On that view, when the population is low, aggregate value is best increased by increasing the total. But as the population grows, adding more people adds far less value, in which case it would be better to increase the average. Such a view would resist the Repugnant Conclusion and say that there is a point where a population, X, is happy enough and large enough such that there is no other population, Y, with very low average well-being that is better than X by virtue of having a much greater population size.
Indeed, the Variable Value view may find its most natural home in the context of playing the role of the group aggregate function in Group Prioritarianism. Hurka [1982] even begins his paper by hinting at the value of having diversity of groups by quoting Thomas Aquinas: ``Just because an angel is better than a stone, it does not follow that two angels is better than an angel and a stone" [1975, III, 71]. However, the Variable Value view alone cannot resist Beneficent Replacement unless it takes group individuation as being normatively significant, and this is something left out of Hurka's picture. Indeed, if supers and humans are treated as one population, the Variable Value view, by itself, would imply that once we have enough happy supers, we should not even allow for any additional humans to exist, even for free, simply because adding more people brings very little value and lowering the average at this point brings a lot of disvalue. But if we take the Variable Value view as playing the role of being the aggregate group welfare function, we avoid this result.
But Variable Value views have their own problems, and indeed, any view that rejects the Repugnant Conclusion have to pick from a wide array of unattractive features [Arrhenius, 2000]. So another way to respond to our problem is to embrace the result that we should benefit The Few over The Masses in this way. After all, from the perspective of insects, they are the Masses and humanity is The Few. If we concoct a view that systematically benefits The Masses over The Few, then it may be the case that all human resources should go instead to benefiting insects instead.
Then again, any view that always benefits The Few over The Masses would be to our disadvantage once the first supers come into existence. For from their perspective, they are The Few and we are The Masses. Thus, humanity is in an awkward position. Views that seek to increase the population of groups with the highest average well-being would result in humanity being crushed by the overwhelming demands from above (i.e. we should give everything to supers). And views that seek to do everything to increase the welfare of groups with the lowest average well being would result in humanity being crushed by the overwhelming demands from below (i.e. we should give everything to insects). I think the view sketched in this paper is a reasonable middle ground. The view may at times give us unintuitive consequences --- but sacrifices must be made for the sake of human survival.
Conclusion:
If super-beneficiaries ever come, the question about whether the human species should be replaced will be the most important question humanity will have to answer. It is no help that all welfarist axiologies until now advocate for human replacement. For this reason, defenders of humanity may think that there are more goods in this world than just welfare, and that diversity is one of them. In this paper, I accommodate this intuition that diversity is valuable without positing diversity as a value. In the axiology developed here, distributions of resources over diverse groups is valuable because well-being is more valuable when given to less well-off groups.
In many ways, however, the view developed in this paper is under-specified. What is presented here is instead a general axiological framework that is capable of resisting Beneficent Replacement. But in order for this axiology to give more concrete guidance on how to distribute resources, we need to specify (1) what groups there are, and (2) how to aggregate group well-being. These are not easy questions to answer. To answer (1), we need to do more metaphysics and sociology. To answer (2), we need to do even more population ethics. Much work needs to be done to fully flesh out Group Prioritarianism, but it is my hope that the framework presented here may be able to absorb the insights from many disparate areas in the humanities to help settle the question of how to best distribute our resources. But whatever the answers to those two questions may be, I'm confident that one recommendation of our framework will still stand: when the supers come, we should not be replaced.
References
Robert Nozick. Anarchy, State, and Utopia. New York: Basic Books, 1974.
Nick Bostrom and Carl Shulman. Sharing the world with digital minds. 2020.
Jacob M. Nebel. Conservatisms about the valuable. Australasian Journal of
Philosophy, 100(1):180–194, 2021. doi: 10.1080/00048402.2020.1861037.
Gerald Allan Cohen. Finding Oneself in the Other. Princeton University Press,
2012.
Samuel Scheffler. Death and the Afterlife. New York, NY: Oup Usa, 2013
Samuel Scheffler. Why Worry About Future Generations? Oxford University
Press, 2018.
Jan Narveson. Moral problems of population. The Monist, 57(1):62–86, 1973.
doi: 10.5840/monist197357134.
Derek Parfit. Reasons and Persons. Oxford, GB: Oxford University Press, 1984.
Thomas Hurka. Value and population size. Ethics, 93(3):496–507, 1982. doi:
10.1086/292462.
Thomas Aquinas. Summa Contra Gentiles. University of Notre Dame Press,
1975.
Gustaf Arrhenius. An impossibility theorem for welfarist axiologies. Economics
and Philosophy, 16(2):247–266, 2000. doi: 10.1017/s0266267100000249.
16
Instead of talking about ``utility" we will talk about well-being for two reasons. The first is that the term ``utility" is often used not as units real well-being, but a function that is used to represent the structure of one's preferences. Secondly, whereas ``utility" is often associated with preference-satisfaction, we want to remain neutral about whether well-being is determined by the satisfaction of one's preferences, or whether it is determined by some pleasurable mental state, or even whether it is determined by the possession of a list of objective goods.
See [Nebel, 2021], [Cohen, 2012], and [Scheffler 2018, 2013] for broadly non-consequentialist reasons to prevent things of value from being replaced by other things of even greater value.
Here I am assuming the Incomparability of Non-Existence which states that, for any person x with positive wellbeing in world w, x is neither better off, worse off, nor equally well off in a world w′ where x does not exist in world w.
For now, I am indifferent to how this aggregate well-being is given. It could just be the total well-being of the group, or even the average. My sense, though, is that since the same motivations for Group Prioritarianism stems from ordinary Prioritarianism, the aggregate well-being function, wi, should itself be a Prioritarian weighted sum function. For simplicity, though, we will assume throughout that the aggregate welfare functions just give us the sum of the wellbeing of the individuals in group i.
This is because we are assuming that a resource that goes to a human can be used to support 10 AI with 100 times more well-being each.
The calculations were made by a simple python script.
For example, some attempt to cash out ``comparative similarity" between worlds in terms of overall space-time likeness, but such an analysis, combined with Lewis's theory of counterfactuals, would imply that the counterfactual ``If the president pressed the nuclear launch button during the Korean War, then the button would have malfunctioned" is true simply because a world where a small malfunction happens is overall more like our world than one where there had been a nuclear bomb detonated during the war. Such an attempt to analyze ``comparative similarity" without reference to counterfactuals themselves is misguided. Rather, one should take from this example that the falsity of the above counterfactual tells us that a world where there is a malfunction is not more similar to actuality than a world where there isn't.