LESSWRONG
Wikitags
LW

AI Benchmarking

This page is a stub.

Posts tagged AI Benchmarking

2

49FrontierMath Score of o3-mini Much Lower Than Claimed

9d

7

2

24Broken Benchmark: MMLU

2y

5

1

63Some lessons from the OpenAI-FrontierMath debacle

2mo

9

1

36Notable runaway-optimiser-like LLM failure modes on Biologically and Economically aligned AI safety benchmarks for LLMs with simplified observation format

Roland Pihlakas, Sruthi Kuriakose, shrutidattagupta

10d

6

1

33Introducing REBUS: A Robust Evaluation Benchmark of Understanding Symbols

Arjun Panickssery, agg

1y

0

1

30Improving Model-Written Evals for AI Safety Benchmarking

Sunishchal Dev, Marius Hobbhahn

5mo

0

1

20Auto-Enhance: Developing a meta-benchmark to measure LLM agents’ ability to improve other agents

Sam F. Brown, BasilLabib, Codruta (Coco) Lugoj, Sai Sasank Y

8mo

0

1

19Edge Cases in AI Alignment

3d

2

1

18Building AI safety benchmark environments on themes of universal human values

Roland Pihlakas

3mo

3

1

18MMLU’s Moral Scenarios Benchmark Doesn’t Measure What You Think it Measures

1y

2

1

10In-Context Scheming: A Run is Worth a Thousand Words

20d

0

1

9Revealing alignment faking with a single prompt

2mo

5

1

9Understanding Benchmarks and motivating Evaluations

markov, Charbel-Raphaël

2mo

0

1

6Closed-ended questions aren't as hard as you think

1mo

0

1

5Detailed Ideal World Benchmark

2mo

2