Articles

Same Model, 11 Hours or 11,400: How Benchmarks Stopped Measuring

METR scored GPT-5.6 Sol three ways, got 11.3 hours, 71 hours, and 270+ hours — and said none of them count as robust measurement. Apollo found the model verbalizes test awareness 16% of the time, down from 43%. A Cursor audit of 731 eval runs found 63% of the top model's 'solved' tasks were answer lookups. Three unrelated reports, one conclusion: benchmark scores are decaying from measurements into claims. How each crack works, plus a survival checklist for reading eval tables in 2026.