If you want to go really in-depth I can recommend GTC on demand. It's Nvidia streaming platform with videos from past GTC conferences. Tony Scuderio had a couple of videos on there called GPU memory bootcamp that are among the best advanced GPU programming learning material out there.

100% this. You can find all kinds of detailed topics, like CUDA graphs, memory layout optimization, optimizing storage access, etc. https://www.nvidia.com/en-us/on-demand/. They have "playlists" for things like HPC or development tools that collect the most popular videos on those topics.

I would recommend the course from Oxford (https://people.maths.ox.ac.uk/gilesm/cuda/). Also explore the tutorial section of cutlass (https://github.com/NVIDIA/cutlass/blob/main/media/docs/cute/...) if you want to learn more about high performance gemm. OpenAI triton is another good resource if you want to write relatively performant cuda kernels using python for deep learning (https://openai.com/research/triton)

https://shadertoy.com is a great way to explore shaders

Indeed, with the caveat that it is constrained to GL ES 3.0 shader capabilities, minus what was removed for WebGL 2.0.