= 27d567bc2d48d9f40e9ccc20ea370b15)
Nick_Beckstead comments on A Proposed Adjustment to the Astronomical Waste Argument - LessWrong
You are viewing a comment permalink. View the original post to see all comments and the full post content.
= 783df68a0f980790206b9ea87794c5b6)
Loading…= b715d02e85f7b3b4ae65ca3d3e450868)
Subscribe to RSS Feed
Powered by Reddit
= f037147d6e6c911a85753b9abdedda8d)
You are viewing a comment permalink. View the original post to see all comments and the full post content.
Comments (38)
Eliezer, I see this post as a response to Nick Bostrom's papers on astronomical waste and not a response to your arguments that FAI is an important cause. I didn't intend for this post to be any kind of evaluation of FAI as a cause or MIRI as an organization supporting that cause. Evaluating FAI as a cause would require lots of analysis I didn't attempt, including:
Whether many of the claims you have made above are true
How effectively we can expect humanity in general to respond to AI risk absent our intervention
How tractable the cause of improving humanity's response is
How much effort is currently going into this cause
Whether the cause could productively absorb additional resources
What our leading alternatives are
My arguments are most relevant to evaluating FAI as a cause for people whose interest in FAI depends heavily on their acceptance of Bostrom's astronomical waste argument. Based on informal conversations, there seem to be a number of people who fall into this category. My own view is that whether FAI is a promising cause is not heavily dependent on astronomical waste considerations, and more dependent on many of these messy details.
Mm, k. I was trying more to say that I got the same sense from your post that Nick Bostrom seems to have gotten at the point where he worried about completely general and perfectly sterile analytic philosophy. Maxipok isn't derived just from the astronomical waste part, it's derived from pragmatic features of actual x-risk problems that lead to ubiquitous threshold effects that define "okayness" - most obviously Parfit's "Extinguishing the last 1000 people is much worse than extinguishing seven billion minus a thousand people" but also including things like satisficing indirect normativity and unfriendly AIs going FOOM. The degree to which x-risk thinking has properly adapted to the pragmatic landscape, not just been derived starting from very abstract a priori considerations, was what gave me that worried sense of overabstraction while reading the OP; and that trigged my reflex to start throwing out concrete examples to see what happened to the abstract analysis in that case.
It may be overly abstract. I'm a philosopher by training and I have a tendency to get overly abstract (which I am working on).
I agree that there are important possibilities with threshold effects, such as extinction and perhaps including your point about threshold effects with indirect normativity AIs. I also think that other scenarios, such as Robin Hanson's scenario, other decentralized market/democracy set-ups, and other scenarios we can't think of are live possibilities. More continuous trajectory changes may be very relevant in these other scenarios.
For what it's worth, I loved this post and don't think it was very abstract. Then again, my background is also in philosophy.