bash - Pipe output from ls to rsync

Question

I'm trying to use rsync to backup folders containing a certain word, using ls, grep and rsync. However rsync does not seem to accept output from grep as input. I've tried the following:

$ ls -d $PWD/** | grep March | rsync -av 'dst'

This does nothing really, even though using just ls -d $PWD/** | grep March produces exactly the list of folders I want to move.

$ ls -d $PWD/** | grep March | xargs -0 | rsync -av 'dst'
$ ls -d $PWD/** | grep March | xargs -0 echo | rsync -av 'dst'
$ ls -d $PWD/** | grep March | xargs -0 rsync -av 'dst'

Many(including dst, here I escape space with \) of the folders contains spaces I thought that might cause problems and found xargs might be of help, but still doesn't move anything.

I have tried the above with sudo, the -avuoption for rsync and -r even though this i included in the -aoption. I usually use the --dry-run option for rsync but I've also tried without. What am I doing wrong? Is it possible to pipe input to rsync like this?

I'm on OSX 10.13.3. GNU bash, version 3.2.57(1)

Thank you.

Answer 1

Using the problematic ls and grep, what you want is likely:

ls -d "$PWD"/** | grep March | xargs -r -n1 -I'{}' rsync -av '{}' 'dst'

A better method is to use --include-from

One option is to let bash emulate a named pipe:

rsync -av0 --include-from=<(find . -path '*March*' -print0) "$PWD"/ /dst/

You can also pipe the find output instead:

find . -path '*March*' -print0| rsync -av0 --include-from=- "$PWD"/ /dst/

-path is used to find "March" anywhere in the filename. (similar to grep)

(rsync might have some parameters to do the filtering itself as well, like the --include and --exclude patterns)

Something like this (untested). See here

rsync -av --include="*/" --include="*March*" --exclude="*" "$PWD"/ /dst/ 

Answer 2

I would suggest creating first your list of files to either include/exclude and then do something like:

rsync -avz --include-from=list.txt source/ destination/

of

rsync -avz --exclude-from=list.txt source/ destination/

To create your list you could use something like:

grep -r March /path > list.txt

Answer 3

I figured it out. Using find to replace ls and grep and piping directly to rsync.

I ended up with the following:

$ find . -type d -maxdepth 1 -name '*March*' -print0 | rsync -av0r --files-from=- ./ /dst/

Here -print0 and -0 'null-terminates' the data as described by Gert van den Berg (I have to because of spaces). The -r seems redundant as it is included in -a but when using --files-from it has to be specified for rsync to sync recursively.

Thank you guys so much, really appreciate it.