Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

rsync: --include-from vs. --exclude-from what is the actual difference?

Tags:

bash

unix

rsync

In the documentation, it mentions these as being files containing lists of either patterns to include or patterns to exclude. However, that implies for inclusions, everything is considered an exclusion except where things match patterns. So for example, an include file containing:

/opt/**.cfg 

Should only include any file named *.cfg that exists anywhere under a directory named opt any where in the tree. So it would match the following:

/opt/etc/myfile.cfg /some/dir/opt/myfile.cfg /notopt/opt/some/other/dir/myfile.cfg 

I'd therefore expect it to implicitly exclude anything else. But that doesn't seem to be the case, since I am seeing this in the itemized output:

*deleting   etc/rc.d/init.d/somescript 

So what is the deal with --include-from and --exclude-from? Are they just aliases for --filter-from?

like image 537
Craig Avatar asked Oct 10 '13 12:10

Craig


People also ask

How does rsync include and exclude work?

Using the include Option As its name implies, this option will filter the files transferred and include files based on the value provided. However, the include option only works along with the exclude option. This is because the default operation for rsync is to include everything in the source directory.

What should I exclude from rsync?

The --exclude-from rsync option allows us to specify a file that contains a list of directories to be excluded. The file should be plaintext, so a simple . txt file will do. List one directory per line as you're compiling the list of directories that you want to exclude.


2 Answers

rsync doesn't work like that. Any file with a filename pattern that does not match any of the include or exclude patterns are considered to be included. In other words, think of the include pattern as a way of overriding exclude pattern.

From the docs (emphasis mine):

Rsync builds an ordered list of include/exclude options as specified on the command line. Rsync checks each file and directory name against each exclude/include pattern in turn. The first matching pattern is acted on. If it is an exclude pattern, then that file is skipped. If it is an include pattern then that filename is not skipped. If no matching include/exclude pattern is found then the filename is not skipped.

So, if you want to include only specific files, you first need to include those specific files, then exclude all other files:

--include="*/" --include="*.cfg" --exclude="*" 

Couple of things to note here:

  1. The include patterns have to come before the excludes, because the first pattern that matches is the one that gets considered. If the file name matches the exclude pattern first, it gets excluded.

  2. You need to either include all subdirectories individually, like --include="/opt" --include="/opt/dir1" etc. for all subdirectories, or use --include="*/" to include all directories (not files). I went with the second option for brevity.

It is quirky and not very intuitive. So read the docs carefully (the "EXCLUDE PATTERNS" section in the link) and use the --dry-run or -n option to make sure it is going to do what you think it should do.

like image 184
Hari Menon Avatar answered Sep 17 '22 14:09

Hari Menon


If you (like me) have a hard time to wrap your head around the FILTER RULES-section in the man-pages but have a basic understanding of find, you could use that instead.

Say you whant to sync everyting with a specific date (ex 2016-02-01) in either the file-name or in a directory-name from /storage/data to rsync_test. Do something like this:

cd /storage/data find . -name '*2016-02-01*' \   | rsync --dry-run -arv --files-from=- /storage/data /tmp/rsync_test 
like image 27
UlfR Avatar answered Sep 21 '22 14:09

UlfR