This is both a blessing (when it actually works on a huge transfer) and a curse (when things are being weird, usually network issues or a flaky remote connection, and you can't remove the weird greyed-out folder).
> and you can't remove the weird greyed-out folder
I'm not sure I've run into this state, but I would guess it would work to toggle your wi-fi off to trip the folder over into the "transfer failed" state (where it's no longer greyed out, and instead displays the little "Retry" emblem on the filename), and then delete it.