While I don't like to argue for LLM competency, you have to remember that at the end of the day LLMs are word generators. They will always be bad at math unless there is a major structural change.

So while they cant learn arithmetic they should be able to learn programming languages given that they are way closer to what it was designed and trained for.