Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There is a cool solution for this: https://huggingface.co/spaces/Jellyfish042/UncheatableEval

This doesn't work for instruction-tuned models, but it's an interesting alternative approach that doesn't need a complicated (and thus gameable) evaluation function or human interaction. Instead, predict the next word with data newer than the training set.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: