rsync: --include-from vs. --exclude-from what is the actual difference?

Question

In the documentation, it mentions these as being files containing lists of either patterns to include or patterns to exclude. However, that implies for inclusions, everything is considered an exclusion except where things match patterns. So for example, an include file containing:

/opt/**.cfg 

Should only include any file named *.cfg that exists anywhere under a directory named opt any where in the tree. So it would match the following:

/opt/etc/myfile.cfg /some/dir/opt/myfile.cfg /notopt/opt/some/other/dir/myfile.cfg 

I'd therefore expect it to implicitly exclude anything else. But that doesn't seem to be the case, since I am seeing this in the itemized output:

*deleting   etc/rc.d/init.d/somescript 

So what is the deal with --include-from and --exclude-from? Are they just aliases for --filter-from?

Answer 1

rsync doesn't work like that. Any file with a filename pattern that does not match any of the include or exclude patterns are considered to be included. In other words, think of the include pattern as a way of overriding exclude pattern.

From the docs (emphasis mine):

Rsync builds an ordered list of include/exclude options as specified on the command line. Rsync checks each file and directory name against each exclude/include pattern in turn. The first matching pattern is acted on. If it is an exclude pattern, then that file is skipped. If it is an include pattern then that filename is not skipped. If no matching include/exclude pattern is found then the filename is not skipped.

So, if you want to include only specific files, you first need to include those specific files, then exclude all other files:

--include="*/" --include="*.cfg" --exclude="*" 

Couple of things to note here:

1. The include patterns have to come before the excludes, because the first pattern that matches is the one that gets considered. If the file name matches the exclude pattern first, it gets excluded.

2. You need to either include all subdirectories individually, like --include="/opt" --include="/opt/dir1" etc. for all subdirectories, or use --include="*/" to include all directories (not files). I went with the second option for brevity.

It is quirky and not very intuitive. So read the docs carefully (the "EXCLUDE PATTERNS" section in the link) and use the --dry-run or -n option to make sure it is going to do what you think it should do.

Answer 2

If you (like me) have a hard time to wrap your head around the FILTER RULES-section in the man-pages but have a basic understanding of find, you could use that instead.

Say you whant to sync everyting with a specific date (ex 2016-02-01) in either the file-name or in a directory-name from /storage/data to rsync_test. Do something like this:

cd /storage/data find . -name '*2016-02-01*' \   | rsync --dry-run -arv --files-from=- /storage/data /tmp/rsync_test