An Exam for AI Scientists
The best AI assistant for scientists scored worse on this exam than a C student on a pop quiz. Just 21% correct answers, and that's the ceiling for the four strongest models. The problems for the new test were written not by programmers but by graduate students: people who deal with real science every day.
The test is called COMPOSITE-STEM.