Randaly

I find the decision to brand the forecast as "AI 2027" very odd. The authors do not in fact believe this; they explicitly give 2028, 2030, or 2033 for their median dates for a superhuman coder. The point of this project was presumably to warn about a possible outcome; by the authors' own beliefs, their warning will be falsified immediately before it is needed. When presenting predictions, forecasters always face tradeoffs regarding how much precision to present. Precise forecasting attracts attempts and motivates action; adding many concrete details produces a compelling story, stimulating discussion; this also involves falsifiable predictions. Emphasizing uncertainty avoids losing credibility when some parts of story inevitably fail; prevents overconfidence; and encourages more robust strategies that can work across a range of outcomes. But I can't think of any reason to only consider a single high precision story that you don't think is all that likely. I think that the excessive precision is pretty important in this case: the current pace of AI R&D spending is unsustainable, so it matters exactly how much more progress is needed for superhuman coders. *** I don't believe that METR's time horizons forecast is sufficiently strong evidence for a precise timeline: 1. *In general* AI competence relative to humans isn't ordered by time- AI's can complete some tasks that would take humans centuries, and can't complete other than would take humans seconds. 2. METR produced their analysis by taking a subset of activities: SWAA, HCAST, and RE-BENCH. SWAA has 66 tasks from 1-30 seconds; HCAST has 97 tasks from 1 minutes to 30 hours; and RE-BENCH has 7 tasks of 8 hours. SWAA and HCAST were created by METR for this research. 3. METR did not- and cannot- use an obvious/straightforward/canonical set of human tasks: current AI's cannot accomplish most human economic tasks, because most tasks do not solely involve interacting with computers. 4. METR created SWAA and HCA