Video streaming is a bit different from "normal" TLS web serving. For video streaming the network is much more likely to be the bottleneck, but traditional TLS web serving tends to send a lot of fairly small packets, and saturating even 40gbe with small packets of TLS is non-trivial.
My experience is that at small (think less than 700 bytes or so) packet sizes, TLS overhead can be a very real cost as you scale up, and it's easy to hit "walls" where really you just need to buy more servers because the engineering cost of getting your servers to really saturate their NICs isn't worth it. How much of a big deal that is for you will depend a lot on exactly what sort of a thing you're serving, though.
I'm curious why TLS packets are small in your experience. We use 16k TLS records, and send at whatever MSS the client has negotatied with us. Thats normally something "normal", like 1420-1460 bytes. Plus we use TSO, so the NIC sees large sends of up to 64KB.
My experience is that at small (think less than 700 bytes or so) packet sizes, TLS overhead can be a very real cost as you scale up, and it's easy to hit "walls" where really you just need to buy more servers because the engineering cost of getting your servers to really saturate their NICs isn't worth it. How much of a big deal that is for you will depend a lot on exactly what sort of a thing you're serving, though.