If you define a new universe with physical laws that are different from our own, performance will also suffer because there is no training data for that custom universe. While some may say this is cheating, a human would be able to deduce correct answers by understanding the deep implications of the new physical laws in that custom universe.