Isn't it easy to train on the specific Exercism exercises that this benchmark uses?

where do you find those o3 high numbers? https://aider.chat/docs/leaderboards/ currently has gemini 2.5 pro as the leader at, as you say, 72.9%.

It's in the OpenAI article post (OP) i.e. OpenAI ran Aider themselves.

Update: the leaderboard has o3 high + 4o tops of the charts now with 82.7%. This is a) amazing b) 20x more expensive than Gemini.

It was a good benchmark until it entered the training set.

thanks