Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Exactly. If I need to move a few megabytes around, <scp -3> and a coffee or a few simple tickets is a good way. A year ago or so, I needed 600GB moved between two systems ASAP during an outage that'd turn into a money-bleed at 6am. If I piped that through the VPN and my workstation, I'd probably still be waiting today.


Some time take a look at lftp [1] and its mirror subsystem for this. It can break up a batch of files or even one large file into multiple SFTP streams. Another upside is that it can replicate most rsync behavior in a SFTP-Only Chroot account. Downside is that without a corresponding daemon like rsync on the other end directory enumeration is slow which isn't a problem if one does not have a complex directory structure.

Play around with the built in rate limit options total and per thread to keep the network people happy.

[1] - https://linux.die.net/man/1/lftp




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: