All of jeffreycaruso's Comments + Replies
Your example of the janitor interrupting the scientist is a good demonstration of my point. I've organized over a hundred cybersecurity events featuring over a thousand speakers and I've never had a single janitor interrupt a talk. On the other hand, I've had numerous "experts" attempt to pass off fiction as fact, draw assumptions from faulty data, and generally behave far worse than any janitor might due to their inflated egos.
Based on my conversations with computer science and philosophy professors who aren't EA-affiliated, and several who are, the...
1I think you're too close to see objectively. I haven't observed any room for policy discussions in this forum that stray from what is acceptable to the mods and active participants. If a discussion doesn't allow for opposing viewpoints, it's of little value. In my experience, and from what I've heard from others who've tried posting here and quit, you have not succeeded in making this a forum where people with opposing viewpoints feel welcome.
Have you read this? https://www.politico.eu/article/rishi-sunak-ai-testing-tech-ai-safety-institute/
"“You can’t have these AI companies jumping through hoops in each and every single different jurisdiction, and from our point of view of course our principal relationship is with the U.S. AI Safety Institute,” Meta’s president of global affairs Nick Clegg — a former British deputy prime minister — told POLITICO on the sidelines of an event in London this month."
"OpenAI and Meta are set to roll out their next batch of AI models imminently. Yet neither has gra...
I hadn't, but I just did and nothing in the article seems to be responsive to what I wrote.
Amusingly, not a single news source I found reporting on the subject has managed to link to the "plan" that the involved parties (countries, companies, etc) agreed to.
Nothing in that summary affirmatively indicates that companies agreed to submit their future models to pre-deployment testing by the UK AISI. One might even say that it seems carefully worded to avoid explicitly pinning the companies down like that.
Yes, I like it! Thanks for sharing that analysis, Gunnar.
Good list. I think I'd use a triangle to organize them. Have consciousness at the base, then sentience, then drawing from your list, phenomenal consciousness, followed by Intentionality?
Thank you for asking.
To generalize across disciplines, a critical aspect of human-level artificial intelligence, requires the ability to observe and compare. This is a feature of sentience. All sentient beings are conscious of their existence. Non-sentient conscious beings exist, of course, but none who could pass a Turing test or a Coffee-making test. That requires both sentience and consciousness.
What happens if you shut down power to the AWS or Azure console powering the Foundation model? Wouldn't this be the easiest way to test various hypotheses associated with the Shutdown Problem in order to either verify it or reject it as a problem not worth sinking further resources into?
That's a good example of my point. Instead of a petition, a more impactful document would be a survey of risks and their probability of occurring in the opinion of these notable public figures.
In addition, there should be a disclaimer regarding who has accepted money from Open Philanthropy or any other EA-affiliated non-profit for research.
Which makes it an existential risk.
"An existential risk is any risk that has the potential to eliminate all of humanity or, at the very least, kill large swaths of the global population." - FLI
What aspect of AI risk is deemed existential by these signatories? I doubt that they all agree on that point. Your publication "An Overview of Catastrophic AI Risks" lists quite a few but doesn't differentiate between theoretical and actual.
Perhaps if you were to create a spreadsheet with a list of each of the risks mentioned in your paper but with the further identification of each as actual or theoretical, and ask each of those 300 luminaries to rate them in terms of probability, then you'd have something a lot more useful.
1I looked at the paper you recommended Zack. The specific section having to do with "how" AGI is developed (para 1.2) skirts around the problem.
"We assume that AGI is developed by pretraining a single large foundation model using selfsupervised learning on (possibly multi-modal) data [Bommasani et al., 2021], and then fine-tuning it using model-free reinforcement learning (RL) with a reward function learned from human feedback [Christiano et al., 2017] on a wide range of computer-based tasks.4 This setup combines elements of the techniques used to tra...
My apologies for not being clear in my Quick Take, Chris. As Zach pointed out in his reply, I posed two issues.
The first being an obvious parallel for me between EA and Judeo-Christian religions. You may or may not agree with me, which is fine. I'm not looking to convince anyone of my point-of-view. I was merely interested in seeing if others here had a similar POV.
The second issue I raised was what I saw as a failure in the reasoning chain where you go from Deep Learning to Consciousness to an AI Armageddon. Why was that leap in faith so compe...
Thank you for the link to that paper, Zack. That's not one that I've read yet.
And you're correct that I raised two separate issues. I'm interested in hearing any responses that members of this community would like to give to either issue.
Are there other forums for AI Alignment or AI Safety and Security besides this one where your article could be published for feedback from perspectives that haven't been shaped by Rationalist thinking or EA?
It would be considerably more difficult, however hacking wasn't really the behavior that I had in mind. Metzinger's BAAN argument goes to the threat of human extinction so I was more curious regarding any research being done regarding how to shut an AI system down with no possibility of a reboot.
I don't see the practical value of a post that starts off with conjecture rather than reality; i.e., "In a saner world...."
You clearly wish that things were different, that investors and corporate executives would simply stop all progress until ironclad safety mechanisms were in place, but wishing doesn't make it so.
Isn't the more pressing problem what can be done in the world that we have, rather than in a world that we wish we had?
Technically, Harry didn't earn his wealth by defeating Voldemort. His mother earned it by giving her life to protect him. It was the sacrifice born of love that defeated the Killing Curse, one of the few ways it could be defeated. Perhaps that's an example of Fundamental Attribution Error.
I just read the post that you linked to. He used the word "prediction" one time in the entire post so I'm having trouble understanding how that was mean't to be an answer to my question. Same with that it's a cornerstone of LessWrong, which, for me, is like asking a Christian why they believe in God, and they answer, because the Bible tells me so.
Is a belief a prediction?
If yes, and a prediction is an act of forecasting, then there must be a way to know if your prediction was correct or incorrect.
Therefore, maybe one requirement for a belief is that ...
Thanks, Trevor. I've bookmarked that link. Just yesterday I started creating a short list of terms for my readers so that link will come in handy.
I've never seen prediction used in reference to a belief before. I'm curious to hear how you arrived at the conclusion that a belief is a prediction.
For me, a belief is something that I cannot prove but I suspect is true whereas a prediction is something that is based in research. Philip Tetlock's book Superforcasting is a good example. Belief has little to nothing to do with making accurate forecasts whereas confronting one's biases and conducting deep research are requirements.
I think it's interesting how the word "belief" can simultaneously ...
Hello, I came across this forum while reading an AI research paper where the authors quoted from Yudkowsky's "Hidden Complexity of Wishes." The linked source brought me here, and I've been reading some really exceptional articles ever since.
By way of introduction, I'm working on the third edition of my book "Inside Cyber Warfare" and I've spent the last few months buried in AI research specifically in the areas of safety and security. I view AGI as a serious threat to our future for two reasons. One, neither safety nor security have ever been p...
What do you estimate the time lag to be if AI startup "Ishtar" utilized your proposed safety method while every other AI startup that Ishtar was competing with in the medical sector, didn't?
It also seems to me like medical terminology would be extremely hard to translate into an ancient language where those words weren't known. The fine-tuning to correct for those errors would have to be calculated into that time lag as well.
Would a bad actor be able to craft Persuasive Adversarial Prompts (PAP) in Sumerian and have the same impact as they did ...
Fired from OpenAI's Superalignment team, Aschenbrenner now runs a fund dedicated to funding AGI-focused startups, according to The Information.
"Former OpenAI super-alignment researcher Leopold Aschenbrenner, who was fired from the company for allegedly leaking information, has started an investment firm to back startups with capital from former Github CEO Nat Friedman, investor Daniel Gross, Stripe CEO Patrick Collision and Stripe president John Collision, according to his personal website.
In a recent podcast interview, Aschenbrenner spoke about the ... (read more)
Leopold's interview with Dwarkesh is a very useful source of what's going on in his mind.
He doesn't believe in a 'sharp left turn', which means he doesn't consider general intelligence to be a discontinuous (latent) capability spike such that alignment becomes significantly more difficult after it occurs. To him, alignment is simply a somewhat harder empirical techniques problem like capabilities work is. I assume he imagines in behavior similar to current RLHF-ed models even as frontier labs have dou... (read more)