Thanks to Jordan Taylor, Mark Xu, Alex Mallen, and Lawrence Chan for feedback on a draft! This post was mostly written by Erik, but we're all currently collaborating on this research direction. Mechanistic anomaly detection (MAD) aims to flag when an AI produces outputs for “unusual reasons.” It is similar...
Epistemic status: Explorative. See results more as a sketch towards a possible issue than a proper derivation of reliable values. Research behind exact numbers is limited and calculations are without error propagation. I'd rather get the idea out there than wait until I find the time to do it properly....
Last Saturday, we played Intelligence Rising. Intelligence Rising is a role-playing game about the global development of transformative AI technologies, where each player or team is a major government or company. We had a great time and we highly recommend you seize any opportunity to play it! What follows is...
Mostly unrelated to the content of the post, but looking at the distributions in this image
this reminds me quite a lot of the anecdote about a Poincaré and the baker.
The anecdote goes:
... (read more)"[...] Poincaré, who made a habit of picking up a loaf of bread each day, noticed after weighing his loaves that they averaged about 950 grams instead of the 1000 grams advertised. He complained to the authorities and afterwards received bigger loaves. Still he had a hunch that something about his bread wasn't kosher. And so with the patience only a famous --- or at least tenured --- scholar can afford, he carefully weighed his bread every day for the next
Another hypothesis: Your description of the task is
the hard parts of application pentesting for LLMs, which are 1. Navigating a real repository of code too large to put in context, 2. Inferring a target application's security model, and 3. Understanding its implementation deeply enough to learn where that security model is broken.
From METR's recent investigation on long tasks you would expect current models not to perform well on this.
I doubt a human professional could do the tasks you describe in something close to an hour, so perhaps its just currently too hard and the current improvements don't make much of a difference for the benchmark, but it might in the future.
(Perhaps you're thinking of this https://www.lesswrong.com/posts/EKu66pFKDHFYPaZ6q/the-hero-with-a-thousand-chances)
Good formulation. "Given it's Monday" can have two different meanings:
So it turns out to 50 % for both but it wasn't initially obvious to me that these two ways would have the same result.
I'd say
The possible observer instances and their probability are:
If I choose a strategy "bet only if blue" (or equivalentely "bet only if red") then expected value for this strategy is so I choose to follow this strategy.
I don't remember what halfer and thirder were or what position I consider to be correct.
Thanks to Jordan Taylor, Mark Xu, Alex Mallen, and Lawrence Chan for feedback on a draft! This post was mostly written by Erik, but we're all currently collaborating on this research direction.
Mechanistic anomaly detection (MAD) aims to flag when an AI produces outputs for “unusual reasons.” It is similar to mechanistic interpretability but doesn’t demand human understanding. The Alignment Research Center (ARC) is trying to formalize “reasons” for an AI’s output using heuristic arguments, aiming for an indefinitely scalable solution to MAD.
As a complement to ARC’s theoretical approach, we are excited about empirical research on MAD. Rather than looking for a principled definition of “reasons,” this means creating incrementally harder MAD benchmarks and better MAD... (read 2760 more words →)
Capabilities leakages don’t really “increase race dynamics”.
Do people actually claim this? Shorter timelines seems like a more reasonable claim to make. To jump directly to impacts on race dynamics is skipping at least one step.
To me it feels like this policy is missing something that accounts for a big chunk of the risk.
While recursive self-improvement is covered by the "Autonomy and replication" point, there is another risk from actors that don't intentionally cause large scale harm but use your system to make improvements to their own systems as they don't follow your RSP. This type of recursive improvement doesn't seem to be covered by any of "Misuse" or "Autonomy and replication".
In short it's about risks due to shortening of timelines.
You can see twin birth rates fell sharply in the late 90s
Shouldn't this be triplet birthrates? Twin birthrates look pretty stable in comparison.
Epistemic status: Explorative. See results more as a sketch towards a possible issue than a proper derivation of reliable values. Research behind exact numbers is limited and calculations are without error propagation. I'd rather get the idea out there than wait until I find the time to do it properly.
This post will sketch at a concern I had recently but also point out one reasons for why it isn't as bad as it may seem.
In short, the insight comes from the fact that amount spent training the largest ML systems scales faster than the spending the median ML researcher spends on training their systems. Thus giving most ML researchers a more and... (read 389 more words →)
Last Saturday, we played Intelligence Rising. Intelligence Rising is a role-playing game about the global development of transformative AI technologies, where each player or team is a major government or company. We had a great time and we highly recommend you seize any opportunity to play it!
What follows is the GM's summary of the game followed by some of our own perspectives and takeaways.
The participants were:
Players learned the rules and how to effectively use their talents and actions. China was an experienced player and began using secret moves with large amounts of resources from... (read 3455 more words →)
These april fools posts are dangerous off-season. With my experience of LessWrong this read as on the tail end of what someone would do, rather than out-of-distribution.