Menu
You can use the following rsync command to easily resume the stopped transfer. Note that the remote server must have rsync installed as well. The -P option is the same as --partial --progress , allowing rsync to work with partially downloaded files. The --rsh=ssh option tells rsync to use ssh as a remote shell.
More specifically, what you can do is to make all destination files "read-only" before scp transfer. This will prevent any existing destination files from being overwritten by scp . After scp transfer is completed, restore the file permissions to the original state.
You can pause a process with SIGSTOP and later continue it with SIGCONT. (See kill -l for a list of signals and use ps to get the process ID).
The reason for scp to stall, is because scp greedily grabs as much bandwith of the network as possible when it transfers files, any delay caused by the network switch of the firewall can easily make the TCP connection stalled.
You should use rsync over ssh
rsync -P -e ssh remoteuser@remotehost:/remote/path /local/path
The key option is -P, which is the same as --partial --progress
By default, rsync will delete any partially transferred file if the transfer is interrupted. In some circumstances it is more desirable to keep partially transferred files. Using the --partial option tells rsync to keep the partial file which should make a subsequent transfer of the rest of the file much faster.
Other options, such -a (for archive mode), and -z (to enable compression) can also be used.
The manual: https://download.samba.org/pub/rsync/rsync.html
An alternative to rsync:
Use sftp with option -r (recursively copy entire directories) and option -a of sftp's get command "resume partial transfers of existing files."
Prerequisite: Your sftp implementation has already a get with -a option.
Example:
Copy directory /foo/bar from remote server to your local current directory. Directory bar will be created in your local
current directory.
echo "get -a /foo/bar" | sftp -r user@remote_server
Since OpenSSH 6.3, you can use reget command in sftp.
It has the same syntax as the get, except that it starts a transfer from the end of an existing local file.
echo "reget /file/path" | sftp -r user@server_name
The same effect has -a switch to the get command or global command-line -a switch of sftp.
Another possibility is to try to salvage the scp you've already started when it stalls.
ctrl+z to background and stop it, then ssh over to the receiving server and login, then exit. Now fg the scp process and watch it resume from 'stalled'!
When rsync stalls as well after couple of seconds when initially running fine I ended up with the following brute force solution to start and stop an re-start the download every 60s:
cat run_me.sh
#!/bin/bash
while [ 1 ]
do
rsync --partial --progress --rsh=ssh user@host:/path/file.tgz file.tgz &
TASK_PID=$!
sleep 60
kill $TASK_PID
sleep 2
done
You can make use of the -rsh and -P options of rsync. -P is for partial download and -rsh indicates transfer is over ssh procotol.
The complete command would be :
rsync -P -rsh remoteuser@remotehost:/remote/path /local/path
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With