Gyrodiot

Message

I'm Jérémy Perret. Based in France. PhD in AI (NLP). AI Safety & EA meetup organizer. Information sponge. Mostly lurking since 2014. Seeking more experience, and eventually a position, in AI safety/governance.

Extremely annoyed by the lack of an explorable framework for AI risk/benefits. Working...

732

153

11y

Gyrodiot

Review of "Fun with +12 OOMs of Compute"

Introduction This review is part of a project with Joe Collman and Jérémy Perret to try to get as close as possible to peer review when giving feedback on the Alignment Forum. Our reasons behind this endeavor are detailed in our original post asking for suggestions of works to review; but the gist is that we hope to bring further clarity to the following questions: * How many low-hanging fruits in terms of feedback can be plucked by getting into a review mindset and seeing the review as part of one’s job? * Given the disparate state of research in AI Alignment, is it possible for any researcher to give useful feedback on any other research work in the field? * What sort of reviews are useful for AI Alignment research? Instead of thinking about these questions in the abstract, we simply make the best review we can, which answers some and gives evidence for others. In this post, we review Fun with +12 OOMs of Compute by Daniel Kokotajlo. We start by summarizing the work, to ensure that we got it right. Then we review the value of the post in itself -- that it, by admitting its hypotheses. We follow by examining the relevance of the work to the field, which hinges on the hypotheses it uses. The last two sections respectively propose follow-up work that we think would be particularly helpful, and discuss how this work fits into the framing of AI Alignment research proposed here by one of us. This post was written by Adam; as such, even if both Joe and Jérémy approve of its content, it’s bound to be slightly biased towards Adam’s perspective. Summary The post attempts to operationalize debates around timelines for Transformative AI using current ML techniques in two ways: by proposing a quantity of resources (compute, memory, bandwidth, everything used in computing) for which these techniques should create TAI with high probability (the +12 OOMs of the title), and by giving concrete scenarios of how the use of these resources could lead to TAI. The operational

65Mar 28, 2021

Gyrodiot

Message

Extremely annoyed by the lack of an explorable framework for AI risk/benefits. Working...

732

153

11y

Review of "Learning Normativity: A Research Agenda"

Introduction We (Adam Shimi, Joe Collman & myself) are trying to emulate peer review feedback for Alignment Forum posts. This is the second review in the series. The first’s introduction sums up our motivation and approach rather well, we will not duplicate it here. Instead, let’s dive into today’s reviewed...

Jun 6, 202137

Review of "Fun with +12 OOMs of Compute"

Mar 28, 202165

Learning from counterfactuals

Alternate title: learning from fictional evidence. I've seen echoes of this idea elsewhere but couldn't find a description that suits me. My main idea is: you can update from your observed reaction to fiction and/or counterfactuals. The fallacy of generalizing from fictional evidence happens when you treat events having happened...

Nov 25, 202011

Mapping Out Alignment

This week, the key alignment group, we answered two questions, 5-minute timer style: 1. Map out all of alignment (25 minutes) 2. Create an image/ table representing alignment (10 min.) You are free to stop here, to actually try to answer the questions yourself. Here is a link for a...

Aug 15, 202043

Resources for AI Alignment Cartography

I want to make an actionable map of AI alignment. After years of reading papers, blog posts, online exchanges, books, and occasionally hidden documents about AI alignment and AI risk, and having extremely interesting conversations about it, most arguments I encounter now feel familiar at best, rehashed at worst. This...

Apr 4, 202046

Layers of Expertise and the Curse of Curiosity

Epistemic status: oversimplification of a process I'm confident about; meant as proof of concept. Related to: Double-Dipping in Dunning-Kruger Expertise comes in different, mostly independent layers. To illustrate them, I will describe the rough process of a curious mind discovering a field of study. Discovery In the beginning, the Rookie...

Feb 12, 201919

Willpower duality

Rationality is designed to make you win, to help you attain your objectives. One of the most prominent phenomena getting in the way is akrasia, the lack of willpower preventing us to perform whatever action we want to do. So, wait, do we want to act or not? There are...

Jan 20, 201710

Load More (7/8)

LESSWRONG
LW

LESSWRONG
LW

Gyrodiot

Gyrodiot

Gyrodiot

Review of "Fun with +12 OOMs of Compute"

Resources for AI Alignment Cartography

Mapping Out Alignment

Review of "Learning Normativity: A Research Agenda"

Gyrodiot

Review of "Learning Normativity: A Research Agenda"

Review of "Fun with +12 OOMs of Compute"

Learning from counterfactuals

Mapping Out Alignment

Resources for AI Alignment Cartography

Layers of Expertise and the Curse of Curiosity

Willpower duality

Review of "Fun with +12 OOMs of Compute"

Resources for AI Alignment Cartography

Mapping Out Alignment

Review of "Learning Normativity: A Research Agenda"

Review of "Learning Normativity: A Research Agenda"

Review of "Fun with +12 OOMs of Compute"

Learning from counterfactuals

Mapping Out Alignment

Resources for AI Alignment Cartography

Layers of Expertise and the Curse of Curiosity

Willpower duality