Review of "Fun with +12 OOMs of Compute"
Introduction This review is part of a project with Joe Collman and Jérémy Perret to try to get as close as possible to peer review when giving feedback on the Alignment Forum. Our reasons behind this endeavor are detailed in our original post asking for suggestions of works to review; but the gist is that we hope to bring further clarity to the following questions: * How many low-hanging fruits in terms of feedback can be plucked by getting into a review mindset and seeing the review as part of one’s job? * Given the disparate state of research in AI Alignment, is it possible for any researcher to give useful feedback on any other research work in the field? * What sort of reviews are useful for AI Alignment research? Instead of thinking about these questions in the abstract, we simply make the best review we can, which answers some and gives evidence for others. In this post, we review Fun with +12 OOMs of Compute by Daniel Kokotajlo. We start by summarizing the work, to ensure that we got it right. Then we review the value of the post in itself -- that it, by admitting its hypotheses. We follow by examining the relevance of the work to the field, which hinges on the hypotheses it uses. The last two sections respectively propose follow-up work that we think would be particularly helpful, and discuss how this work fits into the framing of AI Alignment research proposed here by one of us. This post was written by Adam; as such, even if both Joe and Jérémy approve of its content, it’s bound to be slightly biased towards Adam’s perspective. Summary The post attempts to operationalize debates around timelines for Transformative AI using current ML techniques in two ways: by proposing a quantity of resources (compute, memory, bandwidth, everything used in computing) for which these techniques should create TAI with high probability (the +12 OOMs of the title), and by giving concrete scenarios of how the use of these resources could lead to TAI. The operational
[Fanfiction, continued from yours, wasn't sure if I got the message, please correct me if I went the wrong way]
The master traveler eventually returned to the temple, richer from having successfully led the caravan through and back. He approached the cartographer again, and gave them a small notebook with a nod.
The cartographer's student was confused. "Teacher, have you heard of the other caravan having gone recently through the northern mountains? It seems they didn't make it. Why is that so, when you gave them the same map?"
"I could say," answered the cartographer, "that the earlier one had been unlucky. Wild animals? Bad weather? But I've seen enough travelers to understand the patterns.... (read more)