Yes, I know. But I still cherish that I can do math and math only. (OK, we have to do something like a minor, but I chose math-heavy theoretical computer science to complement.)
I would hate to be stuck knowing/doing only one thing for the rest of my life. As much as i love computers and the internet and the like, I couldn't imagine not being able to pick up a notebook and write a short story or draw something. Not to be able to compose (or hell, even just identify and listen to) good music, discussing politics in depth with lots of details with people and so on.
Not being able to do anything but math/code? That honestly sounds horrible to me.
You are mixing being forced to do something with doing something.
I do a lot of stuff just for fun with other people or without at our own tempo (usually faster). Occasionally I also visit lectures in philosophy. And they would not teach you unicycling, juggling, and walking on a slack rope anyway.
(And it's not math/code in university for me, it's math and math only that I have to do to get my degree.)