Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

How to create a tar file that omits timestamps for its contents?

Is there a way to create a .tar file that omits the values of atime/ctime/mtime for its files/directories?

Why do we want to do this?

We have a step in our build process that generates a directory of artifacts that gets packaged into a tarfile. We expect that build step to be idempotent -- given the same inputs, it produces exactly the same files/output each time.

Ideally, we would like also like the step to be bitwise idempotent across clean builds, so that we can use hashes of successive builds to check that nothing has changed. But because tar files include timestamps (atime/ctime/mtime) for each entry, the tar files created by that build step are never bitwise identical to the previous run, even though the contents of every file inside the archive are bitwise identical.

Is there a way to generate a tarfile that omits the timestamps of its entries, so that the step that generates the archive could be bitwise idempotent? (We want to leverage other file metadata that tar preserves, such as file mode bits and symlinks.)

like image 416
Mickalot Avatar asked Oct 07 '15 16:10

Mickalot


People also ask

Does tar preserve file timestamps?

Using tar -cvf , timestamps are preserved but rounded to the nearest second, i.e. running stat on the files displays only timestamps that end in .

What does the command tar XVZF * .tar do?

The xvf flag is used to extract the files in an archived file. The name of the archived file is sent as an argument to the tar command.

How do I create an archive tar?

To create an archive with tar, use the '-c' (“create”) option, and specify the name of the archive file to create with the '-f' option. It's common practice to use a name with a '. tar' extension, such as 'my-backup. tar'.

How do I create a compressed tar archive?

To create a tar archive, use the -c option followed by -f and the name of the archive. You can create archives from the contents of one or more directories or files. By default, directories are archived recursively unless --no-recursion option is specified.


2 Answers

To have a truly idempotent tar, mtime is a good step but not enough. You also need to set the sort order, the owner and group (together with their mapping) and a proper timezone for mtime (since otherwise you're gonna have issues as well between Mac and Linux).

I ended up with

tar --sort=name --owner=root:0 --group=root:0 --mtime='UTC 2019-01-01' ... | gzip -n 
like image 181
Adracus Avatar answered Sep 28 '22 02:09

Adracus


GNU tar has a --mtime argument, which can be used to store a fixed date in the archive rather than a file's actual mtime:

tar --mtime='1970-01-01' input ... 

When compressing a tarball with gzip, it's also necessary to specify -n to prevent name and timestamp of the tar archive from being stored:

tar --mtime='1970-01-01' input ... | gzip -n >input.tar.gz 
like image 36
Charles Duffy Avatar answered Sep 28 '22 03:09

Charles Duffy