Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The average human did zero studying on representative problems. LLMs did a lot.


I don't know anything about frontiermath problems, but for Putnam problems (which is what the submitted article is about) the average human that takes the exam is an undergraduate mathematics or science major who has studied prior Putnam problems and other similar problems recently to specifically prepare for the exam...and the most common score is still 0.

At top tier schools the most common score will usually be somewhere in the 0 to 10 range (out of a possible 120).


You can get all of the correct numerical answers for the Putnam and still get a zero, because the reasoning is graded very harshly. The scores measured in this paper are not comparable to actual Putnam scores.


Okay? We are measuring capabilities.




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: