Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to perform logging with gsutil rsync

What's the proper way to log any errors or warnings when performing a quiet rsync?

This is what I currently run from my crontab:

gsutil -m -q rsync -r -C /mount1/share/folder gs://my-bucket-1/folder/ > /mount2/share/folder/gsutil.log

Since the log file is always completely empty and I'm uploading terabytes of data I'm starting to think that maybe even errors and warnings are being supressed.

like image 383
fredrik Avatar asked Dec 09 '14 08:12

fredrik


1 Answers

After having realized that this is related to how you pipe stdout and/or stderr to files in general, the answer really lies within this existing thread: How to redirect both stdout and stderr to a file

So a simple solution to log as much as possible into one single log file could be something like:

gsutil -d rsync [src] [dst] &> [logfile]

...where -d enables debug output. I found this to be the only way to show files which were affected by an error such as CommandException: 3 files/objects could not be copied. Please note that -d exposes authentication credentials.

like image 95
fredrik Avatar answered Sep 21 '22 17:09

fredrik