I don't agree that associating multiple related ideas is necessarily 'not good', that said Governance and politics are not the focus of this article. In fact it specifically addresses that.
The article was primarily intended to illustrate how economic alignment may very well be an additional area of alignment and risk that that we should more seriously consider. If you disagree with this, I sincerely want to better understand objections so I'd welcome a thoughtful counter argument that addresses the claims and not the structure of the article.
I'm not clear on what you mean by 'relevant probability', however, yes, I do think that we will see AGI within two decades, and with respect to AI,
P(massive job displacement) is high, perhaps 30-50%,
P(millions die / acute catastrophe) is perhaps 3-6% and
P(billions die / doom) is perhaps 0.2-0.5%
So, I'd say P(catastrophe) is not negligible and will likely slowly rise over time (so long as Information technology generally improves). If it does happened, I would not be surprised if governments take drastic action including potential broad bl...
I'm not actually talking about a complete breakdown of economy or society, just significant shocks to retail, IT, and supply chains and longer term economic shifts.
If I may draw a historical parallel: we shut down our entire airline transportation industry for weeks after 9/11, reducing annual GDP about 0.5%. Not to trivialize the ~3000 deaths and related suffering that occurred but, IMO, AI-facilitated deliberate attacks by malicious actors or nation states could easily be orders of magnitude worse. I honestly don't know exactly how capa...
To preface, i didn't mean to make this a central point and it was mostly directed at families (but also anyone with the interest and means) considering how critical food availability would be in an prolonged AI gone bad scenario when internet connectivity would likely be cut. There are many scenario's where food (and retail at large) could be significantly disrupted due to lack of electronic payment : prolonged power outages, accidental cyber incidents, deliberate cyber attacks, internet shutoffs, and honestly: recognize how significantly our a...
Honestly, some of the mannerisms and vocal features certainly seem inspired by Ms. Johansen's likeness in the Movie Her... and Open AI probably should have not used a voice this close to Johansen's especially in light of her declining the first voice contract offer. But the fact is, just because 4o invokes the spirit of Samantha in "Her", the voice is definitely different and across all the online chatter about GPT-4o last week, I didn't hear about anyone talking specifically about how the voice sounded exactly like Johansen's. It was only after Joha...
Well said. We've been contemplating expanding our family lately and I have to say, I've been secretly thinking many of the same things. That said, if we want humanity to persist and have a chance of one day prospering alongside AI and other technologies to come, children seem like a pretty clear prerequisite (particularly from people like us who care about these bigger pictures). I personally believe there will likely be non-trivial socioeconomic inequality and strife in the wake of AGI, however, I believe that these timescales will be o...
alignment research is currently a mix of different agendas that need more unity. The alignment agendas of some researchers seem hopeless to others, and one of the favorite activities of alignment researchers is to criticize each other constructively
Given the risk-landscape uncertainty and conflicting opinions, I would argue that this is precisely the optimal high-level approach for AI Alignment research agendas at this point in time. 'Casting a broader net' can allow us to more quickly identify and mobilize resources towards areas of urgently-needed alignm...
in short, it seems to me that the crux of the argument comes down to whether there is physiological continuity of self or 'consciousness' for lack of a better word.
I suspect this will also actually have very relevant applications in field such as cryonics which adds an additional layer of complexity because all metabolic processes will completely cease to function.
Conducting the duplication experiment during sleep (or any altered state of consciousness) is interesting but nevertheless there is clearly physical (physiological) continuity of the... (read more)