I haven't yet, but my summer / fall is going to be spent working with one of my profs to set up a lab with CUDA workstations and a Tesla. If you're interested in GPGPU programming, CUDA works with any system with a GeForce 8XXX or higher GPU. So ~$150 for a 9800GTX will get you 128 equivalent cores, which you can easily double at any time by running SLI.
What I'm interested in is seeing how OpenCL plays out, as Khronos seems to have the entire industry behind them (with the exception of Microsoft).
It's pretty nice. As nice as C can get, that is... The biggest hurdle for me was thinking data-parallel as opposed to sequentially or even task-parallel.
I use CUDA for finance based Monte-Carlo simulations. Unless you are at a place with a huge cluster that you have access to, some problems cannot be solved without CUDA.
One downside to CUDA is its massively powerful for single FP precision, but double FP precision performance is less than 1/10 of single FP performance.
When precision is needed CUDA is much less useful, say you're running 10^10 simulations then with single FP precision you will only have a result accurate to 5 significant figures.
