= 27d567bc2d48d9f40e9ccc20ea370b15)
gwern comments on Dr. Jubjub predicts a crisis - Less Wrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (65)
Yes, you'd better have a risk/cost-model.
I avoided mentioning any specific Jubjub problems to try to minimize mind-killing effects. But since you've brought up climate change, which is usually coded as a left-wing issue in our political discourse, I'd just like to mention that there are also Jubjub problems with a very different political coding, e.g. dysgenic pressure, a reactionary right-wing issue.
I'm amused that when I was reading this, it didn't even occur to me that this might be about global warming - I just assumed it was about eugenics.
But fundamentally, I do think that the basic observation is right: our planning horizons should be fairly short, because we just don't know enough about future technology and developments to spend large amounts of resource on things with low option value. There are countless past crises that did not materialize or were averted by other developments; to give an imperfect list off the top of my head: horse shit in the streets of cities, the looming ice age, the degradation of the environment with industrialization, Kessler catastrophe, Y2K, and Hannu Kari's Internet apocalypse.
I am reminded of a story Kelly tells in The Clock of the Long Now about a Scandinavian country which set aside an island for growing big trees for making wooden warships in the 1900s, which was completely wrong since by that point, warships had switched to metal, and so the island became a nature preserve; this is a cute story of how their grossly mistaken forecasts had an unanticipated benefit, but being mistaken is not usually a good way of going about life, and the story would be a lot less cute if the action had involved something more serious like taxation or military drafts or criminal justice or economy-wide regulation.
This was probably Sweden planting lots of oaks in the early 19th century. 34 000 oaks were planted on Djurgården for shipbuilding in 1830. As it takes over a hundred years for the oak to mature, they weren't used and that bit of the Island is now a nature preserve. Quite funny is that when the parliament was deciding this issue, it seems some of the members already doubted whether oak would remain a good material to build ships from for so long.
Also observe that 1900s ≠ 19th century, so they weren't that silly.
Had some trouble finding English references for this, but this (p 4) gives some history and numbers are available in Swedish Wikipedia.
I guess gwern meant the construction was planned to take place in the 1900s.
Quite possible. I didn't intend for that sentence to come across in a hostile way.
Since in Swedish we usually talk about the 1800s and the 1900s instead of the 19th and 20th century, I thought something could have been lost in translation somewhere between the original sources, the book by Kelly and gwern's comment, which is itself ambiguous as to whether it is intended as (set aside an island for growing big trees for making wooden warships) (in the 1900s) or as (set aside an island for growing big trees for (making wooden warships in the 1900s)). (I assumed the former)
As usual, gwern has made a great comment. But I'm going to bite the bullet and come out in favor of the tree plan. Let's go back to the 1830s.
My fellow Swedes! I have a plan to plant 34,000 oak trees. In 120 years we will be able to use them to build mighty warships. My analysis here shows that the cost is modest while the benefits will be quite substantial. But, I hear you say, what if some other material is used to build warships in 120 years? Well, we will always have the option of using the wood to build warships and if we won't take that option it will be because some even better option will have presented itself. That seems like a happy outcome to me. And wood has been useful for thousands of years - it will surely not be completely obsolete in a century. We could always build other things from it, or use it for firewood or designate the forest as a recreational area for esteemed noblemen such as ourselves. Or maybe the future will have some use for forests we cannot yet anticipate [carbon sequestration]. I don't see how we can really go wrong with trees.
Back to the present. I'm concerned with avoiding disasters. "The benefits of this long-term plan were not realized because something even better happened" is only a disaster if the cost of the plan was disastrous. Of course, some people argue that the costs of addressing some of Dr. Jubjub's problems are disastrous and that's something we can discuss on the merits.
Do show your analysis :-) Don't forget about discounting and opportunity costs :-D
For the sake of argument I'm assuming the plan made prima facie sense and was only defeated by technological developments. Sufficiently familiarizing myself with the state of affairs in 1830s Sweden to materially address the question would, I think, be excessively time-consuming.
Correct, but then you shouldn't handwave into existence an assertion which is really at the core of the dispute.
The issue is whether this was a good decision and let's say "good" is defined as low-cost and high-benefit. You are saying "let's assume 'the cost is modest while the benefits will be quite substantial' and then, hey, it's a good decision!".
The argument I am trying to approach is about proposals which make sense under the assumption of little or no relevant technological development but may fail to make sense once disruptive new technology enters the picture. I'm assuming the tree plan made sense in the first way - the cost of planting and tending trees is such and such, the cost of quality wood is such and such and the problems with importing it (our enemies might seek to control the supply) are such and such. Other projects we could spend the same resources on have such and such cost benefit-evaluations of their own. And so on and so forth. In this thought experiment you could assume a very sophisticated analysis which comes up smelling like roses. The only thing it doesn't take into account is disruptive new technology. That's the specific issue I'm trying to address here so that's why I'm willing to assume all the other stuff works for the sake of argument.
In actual history, maybe the tree plan never even made any sense to begin with - maybe wood was cheap and plentiful and planting the oak trees was difficult and expensive. For all I know the whole thing was a ridiculous boondoggle which didn't make sense under any assumption. But that's just an uninteresting case which need not detain us.
One could also see this as part of a diversified investment strategy. Putting aside some existing ressources for future use is surely not a bad idea. The inteded purpose may have been 'wrong'. But as you say: It can have an unanticipated benefit.
And that seeing would be an excellent example of a post-factum justification of an error.
or an argument that we should act so that even if we are in error the consequences are not dire.
I submit that none of us has a clue as to the consequences in a hundred years of what we are doing now.
Really? Is this something you've said before and I've missed it? If true, it has huge implications.
I don't think I've said it before in these words but I may have expressed the same idea.
Why do you think there are huge implications?
If I believe that, I would forget about AI, x-risk and just focus on third-world poverty.
Well, it's up to you to decide how much the uncertainty of outcome should influence your willingness to do something. It's OK to think it's worthwhile to follow a certain path even if you don't know where would it ultimately lead.
You don't have more information about the hundred-year effects of your third-world poverty options than you do about the hundred-year effects of your AI options.
Sure if you intended it for one special purpose and just got lucky with another purpose it would be a good excuse. We don't know what the Scandinavians reasoned other than the possibly often retold war-skip story.
The lesson: If you reserve ressources for a specific purpose either make sure to allow more general usage or reserve multiple different ressources for other purposes too.
And I was more specific-- I thought it was a response to my comment that if you expect a Singularity within a hundred years, you shouldn't be bothering with most eugenics.
It was, in part. But I certainly also had climate change in mind, where I've argued the Jubjub case for years with my friends. I've also seen the "Future tech will make your concerns irrelevant" viewpoint in discussions of resource depletion and overpopulation.