Karl von Wendt

Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell?

On June 22nd, there was a “Munk Debate”, facilitated by the Canadian Aurea Foundation, on the question whether “AI research and development poses an existential threat” (you can watch it here, which I highly recommend). On stage were Yoshua Bengio and Max Tegmark as proponents and Yann LeCun and Melanie Mitchell as opponents of the central thesis. This seems like an excellent opportunity to compare their arguments and the effects they had on the audience, in particular because in the Munk Debate format, the audience gets to vote on the issue before and after the debate. The vote at the beginning revealed 67% of the audience being pro the existential threat hypothesis and 33% against it. Interestingly, it was also asked if the listeners were prepared to change their minds depending on how the debate went, which 92% answered with “yes”. The moderator later called this extraordinary and a possible record for the format. While this is of course not representative for the general public, it mirrors the high uncertainty that most ordinary people feel about AI and its impacts on our future. I am of course heavily biased. I would have counted myself among the 8% of people who were unwilling to change their minds, and indeed I’m still convinced that we need to take existential risks from AI very seriously. While Bengio and Tegmark have strong arguments from years of alignment research on their side, LeCun and Mitchell have often made weak claims in public. So I was convinced that Bengio and Tegmark would easily win the debate. However, when I skipped to the end of the video before watching it, there was an unpleasant surprise waiting for me: at the end of the debate, the audience had seemingly switched to a more skeptical view, with now only 61% accepting an existential threat from AI and 39% dismissing it. What went wrong? Had Max Tegmark and Yoshua Bengio really lost a debate against two people I hadn’t taken very seriously before? Had the whole debate somehow been bia

107Jun 25, 2023

Karl von Wendt

Message

German writer of science-fiction novels and children's books (pen name Karl Olsberg). I blog and create videos about AI risks in German at www.ki-risiken.de and youtube.com/karlolsbergautor.

872

136

What can we learn from parent-child-alignment for AI?

Epistemic status: This is not a scientific analysis, but just some personal observations. I still think they point towards some valid conclusions regarding AI alignment. I am a father of three sons. I would give my life to save each of them without second thoughts. As a father, I certainly...

Oct 29, 202516

Can we ever ensure AI alignment if we can only test AI personas?

When I talk to Claude or ChatGPT, as far as I understand it I’m not really talking to the underlying LLM, but to a fictional persona it selects from the near infinite set of possible personas. If that is true, then when an AI is evaluated, what is really tested...

Mar 16, 202522

The benefits and risks of optimism (about AI safety)

This is a reaction to Nora Belrose’s and Quintin Pope’s AI Optimism initiative. However, others are better qualified to criticize the specific arguments they give for their claim that AI is easy to control. Instead, I will focus on the general stance of optimism, when it can be beneficial and...

Dec 3, 2023-7

The Game of Dominance

Concerns about AI safety rely on the assumption that a sufficiently powerful AI might take control of our future in an undesirable way. Meta’s head of AI, Yann LeCun, correctly points this out in a recent tweet, but then argues that this assumption is wrong because “intelligence” and “dominance” are...

Aug 27, 202324

Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell?

Jun 25, 2023107

A Friendly Face (Another Failure Story)

Edit: Based on the comment by Daniel Kokotajlo, we extended the dialog in the chapter "Takeover from within" by a few lines. The perfect virtual assistant The year is 2026 and the race for human-level artificial general intelligence (AGI) draws to a close. One of the leading AI companies, MegaAI,...

Jun 20, 202365

Agentic Mess (A Failure Story)

This story is also available as a YouTube video. A network of specialized open-source agents emerges Developed by open-source communities, “agentic” AI systems like AutoGPT and BabyAGI begin to demonstrate increased levels of goal-directed behavior. They are built with the aim of overcoming the limitations of current LLMs by adding...

Jun 6, 202346

Load More (7/19)

LESSWRONG
LW

LESSWRONG
LW

Karl von Wendt

Karl von Wendt

Karl von Wendt

Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell?

A Friendly Face (Another Failure Story)

What can we learn from Lex Fridman’s interview with Sam Altman?

VIRTUA: a novel about AI alignment

Karl von Wendt

What can we learn from parent-child-alignment for AI?

Can we ever ensure AI alignment if we can only test AI personas?

The benefits and risks of optimism (about AI safety)

The Game of Dominance

Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell?

A Friendly Face (Another Failure Story)

Agentic Mess (A Failure Story)

Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell?

A Friendly Face (Another Failure Story)

What can we learn from Lex Fridman’s interview with Sam Altman?

VIRTUA: a novel about AI alignment

What can we learn from parent-child-alignment for AI?

Can we ever ensure AI alignment if we can only test AI personas?

The benefits and risks of optimism (about AI safety)

The Game of Dominance

Did Bengio and Tegmark lose a debate about AI x-risk against LeCun and Mitchell?

A Friendly Face (Another Failure Story)

Agentic Mess (A Failure Story)