ESRogs comments on Do Earths with slower economic growth have a better chance at FAI? - Less Wrong

30 Post author: Eliezer_Yudkowsky 12 June 2013 07:54PM

You are viewing a comment permalink. View the original post to see all comments and the full post content.

Comments (174)

You are viewing a single comment's thread. Show more comments above.

Comment author: Eliezer_Yudkowsky 17 June 2013 04:08:00PM 4 points [-]

Increasing capital stocks, improving manufacturing, improving education, improving methodologies for discourse, figuring out important considerations. Making charity more efficient, ending poverty. Improving collective decision-making and governance. All of the social sciences. All of the hard sciences. Math and philosophy and computer science. Everything that everyone is working on, everywhere in the world.

How is an increased capital stock supposed to improve our x-risk / astronomical benefit profile except by being an input into something else? Yes, computer science benefits, that's putatively the problem. We need certain types of math for FAI but does math benefit more from increased capital stocks compared to, say, computing power? Which of these other things are supposed to save the world faster than computer science destroys it, and how? How the heck would terrorism be a plausible input into AI going badly? Terrorists are not going to be the most-funded organizations with the smartest researchers working on AGI (= UFAI) as opposed to MIT, Google or Goldman Sachs.

Does your argument primarily reduce to "If there's no local FOOM then economic growth is a good thing, and I believe much less than you do in local FOOM"? Or do you also think that in local FOOM scenarios higher economic growth now expectedly results in a better local FOOM? And if so is there at least one plausible specific scenario that we can sketch out now for how that works, as opposed to general hopes that a higher economic growth exponent has vague nice effects which will outweigh the shortening of time until the local FOOM with a correspondingly reduced opportunity to get FAI research done in time? When you sketch out a specific scenario, this makes it possible to point out fragile links which conjunctively decrease the probability of that scenario, and often these fragile links generalize, which is why it's a bad idea to keep things vague and not sketch out any concrete scenarios for fear of the conjunction fallacy.

It seems to me that a lot of your reply, going by the mention of things like terrorism and poverty, must be either prioritizing near-term benefits over the astronomical future, or else being predicated on a very different model from local FOOM. We already have a known persistent disagreement on local FOOM. This is an important modular part of the disagreement on which other MIRIfolk do not all line up on one side or another. Thus I would like to know how much we disagree about expected goodness of higher econ growth exponents given local FOOM, and whether there's a big left over factor where "Paul Christiano thinks you're just being silly even assuming that a FOOM is local", especially if this factor is not further traceable to a persistent disagreement about competence of elites. It would then be helpful to sketch out a concrete scenario corresponding to this disagreement to see if it looks even more fragile and conjunctive.

(Note that e.g. Wei Dai also thought it was obviously true that faster econ growth exponents had a negative-sign effect on FAI, though, like me, this debate made him question (but not yet reject) the 'obvious' conclusion.)

Comment author: ESRogs 18 June 2013 11:13:36PM 3 points [-]

(Note that e.g. Wei Dai also thought it was obviously true that faster econ growth exponents had a negative-sign effect on FAI, though, like me, this debate made him question (but not yet reject) the 'obvious' conclusion.)

I'm confused by the logic of this sentence (in particular how the 'though' and 'like me' fit together). Are you saying that you and Wei both at first accepted that faster econ growth meant less chance of FAI, but then were both caused to doubt this conclusion by the fact that others debated the claim?

Comment author: Eliezer_Yudkowsky 18 June 2013 11:37:56PM 2 points [-]

Yep.

Comment author: ESRogs 19 June 2013 12:32:13AM 2 points [-]

This was one of those cases where precisely stating the question helps you get to the answer. Thanks for the confirmation!