LESSWRONG
Wikitags
LW

AI Benchmarking

This page is a stub.

Posts tagged AI Benchmarking

2

61FrontierMath Score of o3-mini Much Lower Than Claimed

1mo

7

2

50Introducing BenchBench: An Industry Standard Benchmark for AI Strength

19d

0

2

24Broken Benchmark: MMLU

2y

5

2

14The real reason AI benchmarks haven’t reflected economic impacts

6d

0

1

69Some lessons from the OpenAI-FrontierMath debacle

3mo

9

1

37Notable runaway-optimiser-like LLM failure modes on Biologically and Economically aligned AI safety benchmarks for LLMs with simplified observation format

Roland Pihlakas, Sruthi Kuriakose, shrutidattagupta

1mo

6

1

33Introducing REBUS: A Robust Evaluation Benchmark of Understanding Symbols

Arjun Panickssery, agg

1y

0

1

30Improving Model-Written Evals for AI Safety Benchmarking

Sunishchal Dev, Marius Hobbhahn

6mo

0

1

20Auto-Enhance: Developing a meta-benchmark to measure LLM agents’ ability to improve other agents

Sam F. Brown, BasilLabib, Codruta (Coco) Lugoj, Sai Sasank Y

9mo

0

1

19Edge Cases in AI Alignment

1mo

3

1

18MMLU’s Moral Scenarios Benchmark Doesn’t Measure What You Think it Measures

2y

2

1

18Building AI safety benchmark environments on themes of universal human values

Roland Pihlakas

4mo

3

1

10In-Context Scheming: A Run is Worth a Thousand Words

1mo

0

1

9Understanding Benchmarks and motivating Evaluations

markov, Charbel-Raphaël

2mo

0

1

9Revealing alignment faking with a single prompt

3mo

5