How to backup with s3cmd, ignoring multiple directories and file types

Question

I've been trying to figure out how to backup the contents of my file server's (CentOS via smb) user's folder, ignoring certain file types and directories. It seems like this should be easy, but I'm not getting anywhere on figuring out how to ignore multiple directories.

I'd like to ignore the following:

• all files and directories starting with a . or a _
• all MS Office temp files (eg ~$*)
• lock files (eg .lock)

I've tried a bunch of different combinations of the --exclude flag, but can't get any to work right.

This is the command that makes the most sense, but it's not excluding anything:

s3cmd sync --dry-run --verbose --delete-removed --exclude '.*' '_*' '~$*' '*.lock' /home/user-folder s3://bucket-name/

Answer 1

If you are already using .gitignore, you can do something like

s3cmd sync --exclude '.git/*' --exclude-from .gitignore <local_dir> s3://<bucket>/

as stated in this blog post and confirmed by the documentation for --exclude-from from the official docs (Ctrl+F and search for "exclude-from").

It works great, with one minor drawback: if you're excluding a folder within .gitignore, you must exclude its contents also, or s3cmd will grab its contents. However, this is easy, you can just add a line like <foldername>/* inside the .gitignore and everything will be ok.

---

EDIT:

Well, better than this. Set up a .s3ignore file and just refer to it from the sync command:

s3cmd sync --exclude-from .s3ignore <local_dir> s3://<bucket>/

.s3ignore example:

.git
.git/*
.gitignore
node_modules
node_modules/*
*.swo
*.swp
*.pyo
*.pyc

Answer 2

I've do something similar. The key is to use --exclude before each pattern you want to match:

s3cmd -v --recursive --exclude ".ts" --exclude ".aac" --exclude "/thumbnails" put /var/www/folder s3://bucket/

Also I managed to use .ts without the wildcard symbol and it worked in my case!