All of alex's Comments + Replies

in short, it seems to me that the crux of the argument comes down to whether there is physiological continuity of self or 'consciousness' for lack of a better word.  

I suspect this will also actually have very relevant applications in field such as cryonics which adds an additional layer of complexity because all metabolic processes will completely cease to function.
 
Conducting the duplication experiment during sleep (or any altered state of consciousness) is interesting but nevertheless there is clearly physical (physiological) continuity of the... (read more)

I don't agree that associating multiple related ideas is necessarily 'not good', that said Governance and politics are not the focus of this article. In fact it specifically addresses that. 

The article was primarily intended to illustrate how economic alignment may very well be an additional area of alignment and risk that that we should more seriously consider.  If you disagree with this, I sincerely want to better understand objections so I'd welcome a thoughtful counter argument that addresses the claims and not the structure of the article. 

 

I'm not clear on what you mean by 'relevant probability', however,  yes, I do think that we will see AGI within two decades, and with respect to AI, 

P(massive job displacement) is high, perhaps 30-50%, 
P(millions die / acute catastrophe) is perhaps 3-6% and
P(billions die / doom) is perhaps 0.2-0.5%

So, I'd say P(catastrophe) is not negligible and will likely slowly rise over time (so long as Information technology generally improves). If it does happened, I would not be surprised if governments take drastic action including potential broad bl... (read more)

1Sherrinford
I did not intend the word 'prepper' to be detogatory, but to be a word for 'classical' preparedness skills. While I understand your risk assessment and it may be true that increasing societal risk makes such prepper skills more valuable, I think it neglects the problem that 'digital' skills, both for job qualifications and for disaster situations, may also become more valuable than before. As time is still only 24 hours a day, it is not clear how the 'life preparedness curriculum' composition should be different compared to, for example, growing up 20 years ago.

I'm not actually talking about a complete breakdown of economy or society, just significant shocks to retail, IT, and supply chains and longer term economic shifts. 

If I may  draw a historical parallel: we shut down our entire airline transportation industry for weeks after 9/11, reducing annual GDP about 0.5%.  Not to trivialize the ~3000 deaths and related suffering that occurred but, IMO, AI-facilitated deliberate attacks by malicious actors or nation states could easily be orders of magnitude worse. I honestly don't know exactly how capa... (read more)

1Sherrinford
I try to summarize your position:  1. You think that with a relevant probability, major catastrophic events will happen that lead to situations in which traditional non-digital "prepper" skills are relevant, 2. and therefore, parents or families should invest a larger share of their own and their children's time and resources into learning such skills, 3. compared to a world that was not "on the eve of AI". Right?

To preface, i didn't mean to make this a central point and it was mostly directed at families (but also anyone with the interest and means) considering how critical food availability would be in an prolonged AI gone bad scenario when internet connectivity would likely be cut.  There are many scenario's where food (and retail at large) could be significantly disrupted due to lack of electronic payment :  prolonged power outages, accidental cyber incidents, deliberate cyber attacks, internet shutoffs, and honestly: recognize how significantly our a... (read more)

1Sherrinford
On the one hand, I understand your point that preparing for breakdown of the economy may be more important if the likelihood of disasters in general increases; even though the most catastrophic AI scenarios would not leave a space to flee to, maybe the likelihood of more mundane disasters also increases? However, it is also possible that the marginal expected value of investing time in such skills goes down. After all, in a more technological society, learning technology skills may be more important than before, so the opportunity cost goes up.

Honestly, some of the mannerisms and vocal features certainly seem inspired by Ms. Johansen's likeness in the Movie Her... and Open AI probably should have not used a voice this close to Johansen's especially in light of her declining the first voice contract offer.  But the fact is, just because 4o invokes the spirit of Samantha in "Her", the voice is definitely different and across all the online chatter about GPT-4o last week, I didn't hear about anyone talking specifically about how the voice sounded exactly like Johansen's. It was only after Joha... (read more)

2Linch
eg https://x.com/karpathy/status/1790373216537502106

Well said.  We've been contemplating expanding our family lately and I have to say, I've been secretly thinking many of the same things.   That said, if we want humanity to persist and have a chance of one day prospering alongside AI and other technologies to come, children seem like a pretty clear prerequisite (particularly from people like us who care about these bigger pictures).  I personally believe there will likely be non-trivial socioeconomic inequality and strife in the wake of AGI, however, I believe that these timescales will be o... (read more)

1Sherrinford
About your "prepper" points, it would be helpful to know the scenario you have in mind here.
1Sherrinford
If we really see AI radically changing everything, why should this assessment still be correct in 10 years? I assume that 30 years ago, people thought the opposite was true. It seems hard to be sure about what to teach children. I do not really see what the uniquely usefull skills of a human will be in 2040 or 2050. Nonetheless, developing these skills as a hobby, without really expecting that to teach something specific as a basic job skill, may be a good idea, also for your point 3.

alignment research is currently a mix of different agendas that need more unity. The alignment agendas of some researchers seem hopeless to others, and one of the favorite activities of alignment researchers is to criticize each other constructively

Given the risk-landscape uncertainty and conflicting opinions, I would argue that this is precisely the optimal high-level approach for AI Alignment research agendas at this point in time. 'Casting a broader net' can allow us to more quickly identify and mobilize resources towards areas of urgently-needed alignm... (read more)