curl's approach will lead to partial and failed downloads. When a client stops accepting new data, servers tend to close the connection after a while.
There are smoother ways to deal with this (i.e. reduce download rate by faking dropped packets to output speed), but if you just want a simple download command, I think both simple solutions are fine.
If the download doesn't fit in RAM, it'll end up swapped out and effectively cached to disk anyway.
The standard solution to this is to write the download to a temporary hidden file on the same volume and then rename it into place once the download succeeds (or delete it on failure).
That's true when downloading to a file, but Invoke-WebRequest is more curl-like than wget-like. It's designed to return an object/struct rather than simply download a file.
If you want to download many/large files, you're probably better off with Start-BitsTransfer.
There are smoother ways to deal with this (i.e. reduce download rate by faking dropped packets to output speed), but if you just want a simple download command, I think both simple solutions are fine.
If the download doesn't fit in RAM, it'll end up swapped out and effectively cached to disk anyway.