General remark: At some point I need to write a post about how I'm worried that there's an "unpacking fallacy" or "conjunction fallacy fallacy" practiced by people who have heard about the conjunction fallacy but don't realize how easy it is to take any event, including events which have already happened, and make it look very improbable by turning one pathway to it into a large series of conjunctions. E.g. I could produce a long list of things which allegedly have to happen for a moon landing to occur, some of which turned out to not be necessary but would look plausible if added to the list ante facto, with no disjunctive paths to the same destination, and thereby make it look impossible. Generally this manifests when somebody writes a list of alleged conjunctive necessities, and I look over the list and some of the items seem unnecessary (my model doesn't go through them at all), obvious disjunctive paths have been omitted, the person has assigned sub-50% probability to things that I see as mainline 90% probabilities, and conditional probabilities when you assume the theory was right about 1-N would be significantly higher for N+1. Most of all, if you imagine taking the negation of the assertion and unpacking it into a long list of conjunctive probabilities, it would look worse - there should be a name for the problem of showing that X has weak arguments but not considering that ~X has even weaker arguments. Or on a meta level, since it is very easy to make things look more conjunctive, we should perhaps not be prejudiced against things which somebody has helpfully unpacked for us into a big conjunction, when the core argument still seems pretty simple on some level.
When I look over this list, my reaction is that:
(1) is a mainline assumption with odds of 5:1 or 10:1 - of course future intergalactic civilization bottlenecks through the goals of a self-improving agency, how would you get to an intergalactic civilization without that happening? If this accounts for much of our disagreement then we're thinking about entirely different scenarios, and I'm not sure how to update from your beliefs about mostly scenario B to my beliefs about mostly scenario A. It makes more sense to call (1) into question if we're really asking about global vs. local, but then we get into the issue of whether global scenarios are mostly automatic losses anyway. If (1) is really about whether we should be taking into account a big chunk of survivable global scenarios then this goes back to a previous persistent disagreement.
(2) I don't see the relevance - why does a long time horizon vs. a short time horizon matter? 80 years would not make me relax and say that we had enough serial depth, though it would certainly be good news ceteris paribus, there's no obvious threshold to cross.
Listing (3) and (4) as separate items was what originally made my brain shout "unpacking fallacy!" There are several subproblems involved in FAI vs. UFAI, of which the two obvious top items are the entire system being conducive to goal stability through self-improvement which may require deducing global properties to which all subsystems must be conducive, and the goal loading problem. These both seem insight-heavy which will require serial time to solve. The key hypothesis is just that there are insight-heavy problems in FAI which don't parallelize well relative to the wide space of cobbled-together designs which might succeed for UFAI. Odds here are less extreme than for (1) but still in the range of 2:1-4:1. The combined 3-4 issue is the main weak point, but the case for "FAI would parallelize better than UFAI" is even weaker.
(5) makes no sense to ask as a conditionally independent question separate from (1); if (1) is true then the only astronomical effects of modern-day economic growth are whatever effects that growth has on AI work, and to determine if economic growth is qualitatively good or bad, we ask about the sign of the effect neglecting its magnitude. I suppose if the effect were trivial enough then we could just increase the planet's growth rate by 5% for sheer fun and giggles and it would have no effect on AI work, but this seems very unlikely; a wealthier planet will ceteris paribus have more AI researchers. Odds of 10:1 or better.
On net, this says that in my visualization the big question is just "Does UFAI parallelize better than FAI, or does FAI parallelize better than UFAI?" and we find that the case for the second clause is weaker than the first; or equivalently "Does UFAI inherently require serial time more than FAI requires serial time?" is weaker than "Does FAI inherently require serial time more than UFAI requires serial time?" This seems like a reasonable epistemic state to me.
The resulting shove at the balance of the sign of the effect of economic growth would have to be counterbalanced by some sort of stronger shove in the direction of modern-day economic growth having astronomical benefits. And the case for e.g. "More econ growth means friendlier international relations and so they endorse ideal Y which leads them to agree with me on policy Z" seems even more implausible when unpacked into a series of conjunctions. Lots of wealthy people and relatively friendly nations right now are not endorsing policy Z.
To summarize and simplify the whole idea, the notion is:
Right now my estimate of the sign of the astronomical effect of modern-day economic growth is dominated by a 2-node conjunction of, "Modern-day econ growth has a positive effect on resources into both FAI and UFAI" and "The case for FAI parallelizing better than UFAI is weaker than the converse case". For this to be not true requires mainly that somebody else demonstrate an effect or set of effects in the opposite direction which has better net properties after its own conjunctions are taken into account. The main weakness in the argument and lingering hope that econ growth is good, isn't that the original argument is very conjunctive, but rather it's that faster econ growth seems like it should have a bunch of nice effects on nice things and so the disjunction of other econ effects might conceivably swing the sign the other way. But it would be nice to have at least one plausible such good effect without dropping our standards so low that we could as easily list a dozen equally (im)plausible bad effects.
Even without doing any calculations, it is extraordinarily hard to imagine that the difference between "world at war" and "world at peace" is less than the difference between "world with slightly more parallelization in AI work" and "world with slightly less parallelization;"
With small enough values of 'slightly' obviously the former will have a greater effect, the question is the sign of that effect; also it's not obvious to me that moderately lower amounts of econ growth lead to world wars, and war seems qualitatively different in many respects from poverty. I also have to ask if you are possibly maybe being distracted by the travails of one planet as a terminal value, rather than considering that planet's instrumental role in future galaxies.
At some point I need to write a post about how I'm worried that there's an "unpacking fallacy" or "conjunction fallacy fallacy" practiced by people who have heard about the conjunction fallacy but don't realize how easy it is to take any event, including events which have already happened, and make it look very improbable by turning one pathway to it into a large series of conjunctions.
Related: There's a small literature on what Tversky called "support theory," which discusses packing and unpacking effects: Tversky & Ko...
I was raised as a good and proper child of the Enlightenment who grew up reading The Incredible Bread Machine and A Step Farther Out, taking for granted that economic growth was a huge in-practice component of human utility (plausibly the majority component if you asked yourself what was the major difference between the 21st century and the Middle Ages) and that the "Small is Beautiful" / "Sustainable Growth" crowds were living in impossible dreamworlds that rejected quantitative thinking in favor of protesting against nuclear power plants.
And so far as I know, such a view would still be an excellent first-order approximation if we were going to carry on into the future by steady technological progress: Economic growth = good.
But suppose my main-line projection is correct and the "probability of an OK outcome" / "astronomical benefit" scenario essentially comes down to a race between Friendly AI and unFriendly AI. So far as I can tell, the most likely reason we wouldn't get Friendly AI is the total serial research depth required to develop and implement a strong-enough theory of stable self-improvement with a possible side order of failing to solve the goal transfer problem. Relative to UFAI, FAI work seems like it would be mathier and more insight-based, where UFAI can more easily cobble together lots of pieces. This means that UFAI parallelizes better than FAI. UFAI also probably benefits from brute-force computing power more than FAI. Both of these imply, so far as I can tell, that slower economic growth is good news for FAI; it lengthens the deadline to UFAI and gives us more time to get the job done. I have sometimes thought half-jokingly and half-anthropically that I ought to try to find investment scenarios based on a continued Great Stagnation and an indefinite Great Recession where the whole developed world slowly goes the way of Spain, because these scenarios would account for a majority of surviving Everett branches.
Roughly, it seems to me like higher economic growth speeds up time and this is not a good thing. I wish I had more time, not less, in which to work on FAI; I would prefer worlds in which this research can proceed at a relatively less frenzied pace and still succeed, worlds in which the default timelines to UFAI terminate in 2055 instead of 2035.
I have various cute ideas for things which could improve a country's economic growth. The chance of these things eventuating seems small, the chance that they eventuate because I write about them seems tiny, and they would be good mainly for entertainment, links from econblogs, and possibly marginally impressing some people. I was thinking about collecting them into a post called "The Nice Things We Can't Have" based on my prediction that various forces will block, e.g., the all-robotic all-electric car grid which could be relatively trivial to build using present-day technology - that we are too far into the Great Stagnation and the bureaucratic maturity of developed countries to get nice things anymore. However I have a certain inhibition against trying things that would make everyone worse off if they actually succeeded, even if the probability of success is tiny. And it's not completely impossible that we'll see some actual experiments with small nation-states in the next few decades, that some of the people doing those experiments will have read Less Wrong, or that successful experiments will spread (if the US ever legalizes robotic cars or tries a city with an all-robotic fleet, it'll be because China or Dubai or New Zealand tried it first). Other EAs (effective altruists) care much more strongly about economic growth directly and are trying to increase it directly. (An extremely understandable position which would typically be taken by good and virtuous people).
Throwing out remote, contrived scenarios where something accomplishes the opposite of its intended effect is cheap and meaningless (vide "But what if MIRI accomplishes the opposite of its purpose due to blah") but in this case I feel impelled to ask because my mainline visualization has the Great Stagnation being good news. I certainly wish that economic growth would align with FAI because then my virtues would align and my optimal policies have fewer downsides, but I am also aware that wishing does not make something more likely (or less likely) in reality.
To head off some obvious types of bad reasoning in advance: Yes, higher economic growth frees up resources for effective altruism and thereby increases resources going to FAI, but it also increases resources going to the AI field generally which is mostly pushing UFAI, and the problem arguendo is that UFAI parallelizes more easily.
Similarly, a planet with generally higher economic growth might develop intelligence amplification (IA) technology earlier. But this general advancement of science will also accelerate UFAI, so you might just be decreasing the amount of FAI research that gets done before IA and decreasing the amount of time available after IA before UFAI. Similarly to the more mundane idea that increased economic growth will produce more geniuses some of whom can work on FAI; there'd also be more geniuses working on UFAI, and UFAI probably parallelizes better and requires less serial depth of research. If you concentrate on some single good effect on blah and neglect the corresponding speeding-up of UFAI timelines, you will obviously be able to generate spurious arguments for economic growth having a positive effect on the balance.
So I pose the question: "Is slower economic growth good news?" or "Do you think Everett branches with 4% or 1% RGDP growth have a better chance of getting FAI before UFAI"? So far as I can tell, my current mainline guesses imply, "Everett branches with slower economic growth contain more serial depth of cognitive causality and have more effective time left on the clock before they end due to UFAI, which favors FAI research over UFAI research".
This seems like a good parameter to have a grasp on for any number of reasons, and I can't recall it previously being debated in the x-risk / EA community.
EDIT: To be clear, the idea is not that trying to deliberately slow world economic growth would be a maximally effective use of EA resources and better than current top targets; this seems likely to have very small marginal effects, and many such courses are risky. The question is whether a good and virtuous person ought to avoid, or alternatively seize, any opportunities which come their way to help out on world economic growth.
EDIT 2: Carl Shulman's opinion can be found on the Facebook discussion here.