rsync'ing files between two remote servers, get errors stating rsync command not found on remote server

Question

I'm trying to rsync files between two servers with

rsync -avlzp /source user@server:/destination

but instead I get errors stating

bash: rsync: command not found
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(635) [sender=3.0.2]

but rsync is installed on both servers. What am I doing wrong? I've also tried

rsync -av -e "ssh -l ssh-user" /source server:/destination

with the same result.

I'm mainly trying to use rsync so that it only copies over differences if they exist...

Thanks

Answer 1

rsync may be in the path on the remote machine when running an interactive shell, but not when running rsync by ssh (with a non-interactive shell). .bashrc is normally only run for interactive shells, for example, so if your path is defined there, it will not be parsed when running rsync --server on the remote end. Try /etc/profile. See bash(1) for more details.

You can use --rsync-path to specify the path to the rsync binary on the remote machine.

Answer 2

There is a lot of good information around about this problem, including the answer above. But in my case, I am an administrator on both the source and destination machines, and I wished to understand what was going on and what "right" configuration options would make this problem go away. I haven't been entirely successful but it seems likely that others may find this information useful.

As the answer above states, rsync may be in the path on the remote machine when running an interactive shell, but not when running rsync by ssh. As Siddesh notes at http://siddesh-bg.blogspot.com/2009/02/rsync-command-not-found-error-even.html this can be solved by explicitly specifying the path to the remote rsync from your local shell:

rsync -av --rsync-path=/usr/local/bin/rsync -e "ssh -l ssh-user" /source server:/destination

But as an administrator I wanted to fix the problem, not just work around it. OldSchool posted a useful troubleshooting approach at https://groups.google.com/group/comp.unix.solaris/browse_thread/thread/0a06a4d382d752d8?pli=1

He suggest you find out what shell is being run for you on the remote machine, then what PATH has been established there:

ssh user@host 'echo $SHELL' 
ssh user@host 'echo $PATH' 

Tom Feiner has a nice post at Why does an SSH remote command get fewer environment variables then when run manually? that discusses the environment establishment of the SSH command execution shell.

OldSchool also noted that "man sshd_config" provides some information about how an ssh session gets an environment. The sshd_config setting "PermitUserEnvironment" can be set to allow a user's ~/.ssh/environment on the server side and environment options in the AuthorizedKeysFile file to be processed by sshd. The default is no.

So with default sshd_config settings the path is going to be built by your default shell. If your ssh command requests an interactive session (ssh user@host) then on the remote server the default shell will do an interactive login, which will probably result in the PATH that you expect. But if you are initiating an ssh session for rsync (rsync -av --rsync-path=/usr/local/bin/rsync -e "ssh -l ssh-user" /source server:/destination) then the remote server gives you a SSH command execution shell that is a non-interactive shell. In my case, the remote non-interactive shell (bash) is not executing /etc/profile, because it only does that for interactive shells or non-interactive shells with the --login option. I could force it like this:

ssh user@host 'echo $PATH;source /etc/profile;echo $PATH' 

I also studied Daniel Barrett & Richard Silverman's "SSH The Secure Shell" (O'Reilly 2001) and set a PATH in /etc/ssh/sshrc that I hoped would make rsync available to SSH command execution shells, but, /etc/ssh/sshrc is executed before the users shell or command is invoked, and apparently it's environment is not passed on to that shell or command.

At this point I am going to be satisfied with the rsync command line option, i.e.:

rsync -av --rsync-path=/usr/local/bin/rsync -e "ssh -l ssh-user" /source server:/destination

because I am concerned that if I change the sshd_config setting "PermitUserEnvironment" then I will probably fail a security audit on my servers. I may revisit this after I have more experience with our organization's security audits.