EuanMcLean

Even Superhuman Go AIs Have Surprising Failure Modes

In March 2016, AlphaGo defeated the Go world champion Lee Sedol, winning four games to one. Machines had finally become superhuman at Go. Since then, Go-playing AI has only grown stronger. The supremacy of AI over humans seemed assured, with Lee Sedol commenting they are an "entity that cannot be defeated". But in 2022, amateur Go player Kellin Pelrine defeated KataGo, a Go program that is even stronger than AlphaGo. How? It turns out that even superhuman AIs have blind spots and can be tripped up by surprisingly simple tricks. In our new paper, we developed a way to automatically find vulnerabilities in a "victim" AI system by training an adversary AI system to beat the victim. With this approach, we found that KataGo systematically misevaluates large cyclically connected groups of stones. We also found that other superhuman Go bots including ELF OpenGo, Leela Zero and Fine Art suffer from a similar blindspot. Although such positions rarely occur in human games, they can be reliably created by executing a straightforward strategy. Indeed, the strategy is simple enough that you can teach it to a human who can then defeat these Go bots unaided. The victim and adversary take turns playing a game of Go. The adversary is able to sample moves the victim is likely to take, but otherwise has no special powers, and can only play legal Go moves. Our AI system (that we call the adversary) can beat a superhuman version of KataGo in 94 out of 100 games, despite requiring only 8% of the computational power used to train that version of KataGo. We found two separate exploits: one where the adversary tricks KataGo into passing prematurely, and another that involves coaxing KataGo into confidently building an unsafe circular group that can be captured. Go enthusiasts can read an analysis of these games on the project website. Our results also give some general lessons about AI outside of Go. Many AI systems, from image classifiers to natural language processing systems, are v

130Jul 20, 2023

EuanMcLean

Message

560

Computational functionalism probably can't explain phenomenal consciousness

I’ve updated quite hard against computational functionalism (CF) recently (as an explanation for phenomenal consciousness), from ~80% to ~30%. Of course it’s more complicated than that, since there are different ways to interpret CF and having credences on theories of consciousness can be hella slippery. So far in this sequence,...

Dec 10, 202418

Do simulacra dream of digital sheep?

This is the third in a sequence of posts scrutinizing computational functionalism (CF). In a previous post, I defined a concrete claim that computational functionalists tend to make: > Theoretical CF: A simulation of a human brain on a computer, with physics perfectly simulated down to the atomic level, would...

Dec 3, 202416

Is the mind a program?

This is the second in a sequence of posts scrutinizing computational functionalism (CF). In my last post, I defined a concrete claim that computational functionalists tend to make: > Practical CF: A simulation of a human brain on a classical computer, capturing the dynamics of the brain on some coarse-grained...

Nov 28, 20247

Two flavors of computational functionalism

This is intended to be the first in a sequence of posts where I scrutinize the claims of computational functionalism (CF). I used to subscribe to it, but after more reading, I’m pretty confused about whether or not it’s true. All things considered, I would tentatively bet that computational functionalism...

Nov 25, 202427

Searching for phenomenal consciousness in LLMs: Perceptual reality monitoring and introspective confidence

To update our credence on whether or not LLMs are conscious, we can ask how many of the Butlin/Long indicator properties for phenomenal consciousness are satisfied by LLMs. To start this program, I zoomed in on an indicator property that is required for consciousness under higher-order theory, nicknamed “HOT-2”: Metacognitive...

Oct 29, 202447

What mistakes has the AI safety movement made?

This is the third of three posts summarizing what I learned when I interviewed 17 AI safety experts about their "big picture" of the existential AI risk landscape: how AGI will play out, how things might go wrong, and what the AI safety community should be doing. See here for...

May 23, 202465

What should AI safety be trying to achieve?

This is the second of three posts summarizing what I learned when I interviewed 17 AI safety experts about their "big picture" of the existential AI risk landscape: how will artificial general intelligence (AGI) play out, how things might go wrong, and what the AI safety community should be doing....

May 23, 202417

Load More (7/14)

LESSWRONG
LW

LESSWRONG
LW

EuanMcLean

EuanMcLean

EuanMcLean

Even Superhuman Go AIs Have Surprising Failure Modes

AI Safety in a World of Vulnerable Machine Learning Systems

What mistakes has the AI safety movement made?

Deconfusing Landauer's Principle

EuanMcLean

Computational functionalism probably can't explain phenomenal consciousness

Do simulacra dream of digital sheep?

Is the mind a program?

Two flavors of computational functionalism

Searching for phenomenal consciousness in LLMs: Perceptual reality monitoring and introspective confidence

What mistakes has the AI safety movement made?

What should AI safety be trying to achieve?

Even Superhuman Go AIs Have Surprising Failure Modes

AI Safety in a World of Vulnerable Machine Learning Systems

What mistakes has the AI safety movement made?

Deconfusing Landauer's Principle

Computational functionalism probably can't explain phenomenal consciousness

Do simulacra dream of digital sheep?

Is the mind a program?

Two flavors of computational functionalism

Searching for phenomenal consciousness in LLMs: Perceptual reality monitoring and introspective confidence

What mistakes has the AI safety movement made?

What should AI safety be trying to achieve?