Here are my thoughts having just read the summary above, not the whole essay yet.
They take the fundamental rules of existence and the human condition (the “existential status quo”) as a given and don’t try to change it.
This sentence confused me. I think it could be fixed with some examples of what would constitute an instance of challenging the "existential status quo" in action. The first example I was thinking of would be ending death or aging, except you've already got transhumanists in there.
Other examples might include: * mitigating existential risks * suggesting and working on civilization as a whole reaching a new level, such as colonizing other planets and solar systems. * trying to implement better design for the fundamental functions of ubiquitous institutions, such as medicine, science, or law.
Again, I'm just giving quick feedback. Hopefully you've already given more detail in essay. Other than that, your summary seems fine to me.
Subscribe to RSS Feed
= f037147d6e6c911a85753b9abdedda8d)
But what does one maximize?
We can not maximize more than one thing (except in trivial cases). It's not too hard to call the thing that we want to maximize our utility, and the balance of priorities and desires our utility function. I imagine that most of the components of that function are subject to diminishng returns, and such components I would satisfice. So I understand this whole thing as saying that these things have the potential for unbounded linear or superlinear utility?
I'm not sure if I'm confused.
Expected utility :)
I guess I have to disagree. Sure, in any given moment you can maximize only one thing but this is simply not true for larger time horizons. Let's illustrate this with a typical day of Imaginary John: He wakes up and goes to work at an investment bank to earn money (money maximizing) to donate it later to GiveWell (ethical maximizing). Later at night he goes on OKCupid/or to a party to find his true soulmate (romantic maximizing). He maximized three different things in just one day. But I agree that there are always trade-offs. John could had worked all day instead of going to the party.
I think that many components of my utility function are not subject to diminishing returns. Let's use your first example, "epistemic rationality". Epistemic rationality is basically about acquiring true beliefs or new (true) information. But sometimes learning new information can radically change your whole life and thus is not subject to diminishing marginal returns. To use an example: Let's imagine you are a consequentialist and donate to charities to help blind people in the USA. Then you learn about effective altruism and cost-effectiveness and decide to donate to the most effective charities. Reading such arguments has just increased your positive impact on the world by a hundredfold! (Btw, Bostrom uses the term "crucial consideration" exactly for such things.) One could make the same argument for, say, AGI programmers reading "Superintelligence" for the first time. Given that they understand this new information, it will probably be more useful to them than most of the stuff they've learnt before.
On to the next issue – Ethics: Let's say one value of mine is to reduce suffering (what could be called non-suffering maximizing). This value is also not subject to diminishing marginal returns. For example, imagine 10.000 people getting tortured (sorry). Saving the first 100 people from getting tortured is as valuable to me as saving the last 100 people.
Admittedly, with regards to social interactions there is probably an upper bound somewhere. But this upper bound is probably much higher than most seem to assume. Also, it occurred to me that one has to distinguish between the quality and the quantity of one's social interactions. The quality of one's social interactions is unlikely to be subject to diminishing marginal returns any time soon. However, the quantity of social interactions definitely is subject to diminishing marginal returns (see e.g. Dunbar's number).
Btw, "attention" is another resource that actually has increasing marginal returns (I've stolen this example from Valentine Smith who used it in a CFAR workshop).
But I agree that unbounded utility functions can be problematic (but bounded ones, too.) However, satisficing might not help you with this.