LESSWRONG
LW

All of Mordechai Rorvig's Comments + Replies

Can someone, anyone, make superintelligence a more concrete concept?
Mordechai Rorvig10

Thanks - didn't see his remarks about this, specifically. I'll try to look them up, thanks.

Reply
If Neuroscientists Succeed
Mordechai Rorvig80

I can see what you mean. However, I would say that just claiming "that's not what we are trying to do" is not a strong rebuttal. For example, we would not accept such a rebuttal from a weapons company, which was seeking to make weapons technology widely available without regulation. We would say - it doesn't matter how you are trying to use the weapons, it matters how others are, with your technology.

In the long term, it does seem correct to me that the greater concern is issues around superintelligence. However, in the near term it seems the issue is we a... (read more)

Reply
If Neuroscientists Succeed
Mordechai Rorvig10

Thank you for the feedback. Did you know of any similar writing making similar points that were more readable, in your mind? What was an example of a place that you found it meandering or overlong? This could help me improve future drafts. I appreciate your interest, and I'm sorry you felt it wasn't concise and was overly 'vibey.'

Reply
how do the CEOs respond to our concerns?
Answer by Mordechai Rorvig32

I don't know, but I would suspect that Sam Altman and other OpenAI staff have strong views in favor of what they're doing. Isn't there probably some existing commentary out there on what he thinks? I haven't checked. But also, it is problematic to assume that science is rational. It isn't, and many people often hold differing views up to and including the time that something becomes incontrovertibly established.

Further, an issue here is when someone has a strong conflict of interest. If a human is being paid millions of dollars per year to pursue their cur... (read more)

Reply
1KvmanThinking
The difference is that if the Exxon Mobil CEO internalizes that (s)he is harming the environment, (s)he has to go and get a completely new job, probably building dams or something. But if Sam Altman internalizes that he is increasing our chance of extinction, all he has to do is tell all his capability researchers to work on alignment, and money is still coming in; only now, less of it comes from ChatGPT subscriptions and more of it comes from grants from the Long-Term Future Fund. It's a much easier and lighter shift. Additionally, he knows that he can go back to working on capabilities once he's reasonably sure that he's not going to kill everyone, while the Exxon Mobil CEO knows (s)he can never drill again.  If we frame this as a little indefinite pause that all the tech companies undertake at the same time, to allow them to chill, work on safety, smell the roses, and seem all wise and understanding to the general public, then I think they would be on board. Although the rich and powerful are not exactly known for being able to update their beliefs.
Can someone, anyone, make superintelligence a more concrete concept?
Mordechai Rorvig20

Honestly, I think the journalism project I was working on in the last year may be most important for the way it sheds light on your question.

The purpose of the project, to be as concise as possible, was to investigate the evidence from neuroscience that there may be a strong analogy between modern AI programs, like language models, and distinctive subregions of the brain, like the so-called language network, responsible for our abilities to process and generate language.

As such, if you imagine coupling such a frontier language model with a crude agent arch... (read more)

Reply
2Ori Nagel
Really interesting project, Mordechai! Have you seen some of Geoffrey Hinton's latest remarks? He's said some things along these lines actually. Feel free to message me and I can point you to it.