Exactly, 44100hz is pretty good and minidisk, while lossy, was way way better than tapes... hell, even Chrome tapes couldn't properly store frequencies higher than 16khz.
Anything above 44100hz is just headroom for mixing.
And listening to stuff at 96khz or anything like that is the audiophile version of buying gold plated hdmi cables.
Yes, for playback. For some people there's a lot of confusion about bit depth since it can represent a range of integers (like how the data is stored in a wav file or what dacs process) or a range of floats.
Internally, audio apps typically represent signals as sequences of 32bit floats between -1 and +1.
Headroom has nothing to do with sampling frequency (bit depth do). But it has been explained here already.
While you're absolutely right that listening to stuff at 96khz is pointless, it's worth noting that these high sampling frequencies are useful for other audio applications that are not on the consumer side (like recording, mixing, instruments sampling, and more).
Anything above 44100hz is just headroom for mixing.
And listening to stuff at 96khz or anything like that is the audiophile version of buying gold plated hdmi cables.