Rsync
rsync is a command-line utility for fast, incremental file transfer and synchronization. It is ideal for advanced users or automated scripts moving data between local systems and HPC clusters.
ℹ Key Features: Efficient (transfers only changed parts of files), preserves permissions, timestamps, and links, and is available on all SCU clusters.
Network Requirements
To use rsync with a remote cluster, you must have SSH access. If you are not on the campus network, you must connect to the Weill VPN first.
How to Use Rsync
For long-running transfers, we highly recommend running rsync inside a screen session to ensure the process continues even if your connection drops.
Copy Local Files to Remote (Upload)
From your terminal, use the following syntax:
rsync -av laptopFromDir/ loginid@scu-login01.med.cornell.edu:~/loginToDir/
⚠️ The Trailing Slash Rule:
- dir/ = copy the contents of the directory
- dir = copy the directory itself
Copy Remote Files to Local (Download)
rsync -av loginid@scu-login01.med.cornell.edu:~/loginToDir/ laptopFromDir/
Copy a Directory Locally
rsync -a /path/to/source/directory/ /path/to/destination/directory/
Common Rsync Flags
| Flag | Description | Example |
|---|---|---|
| -a | Archive Mode: preserves attributes like permissions and times | rsync -a src/ dest/ |
| -v | Verbose: provides detailed terminal output during transfer | rsync -av src/ dest/ |
| -z | Compress: compresses data data during transfer to save bandwidth | rsync -az src/ dest/ |
| -P | Shows dynamic progress bars and allows resuming interrupted transfers | rsync -aP src/ dest/ |
| -n | Dry Run: previews files that would transfer without writing changes | rsync -anv src/ dest/ |
| --delete | Removes files from target directory that do not exist in source | rsync -a --delete src/ dest/ |
| --update | Skips updating files that are newer on the target system | rsync -av --update src/ dest/ |