How can I set file creation times in ZFS?

Question

I've just got a NAS running ZFS and I'd like to preserve creation times when transferring files into it. Both linux/ext4 (where the data is now) and zfs store creation time or birth time. In the case of zfs it's even reported by the stat command. But I haven't been able to figure out how I can set the creation time of a file so it mirrors the creation time in the original file system. Unlike an ext4->ext4 transfer where I can feed debugfs a script to set the file creation times.

Is there a tool similar to debugfs for ZFS?

PS. To explain better:

• I have a USB drive attached to a Ubuntu 14.04 laptop. It holds a file system where I care about the creation date (birth date) of the individual files. I consult these creation timestamps often using a script based on debugfs, which reports it as crtime.

• I want to move the data to a NAS box running ZFS, but the methods I know (scp -p -r, rsync -a, and tar, among others I've tried) preserve the modification time but not the creation time.

• If I were moving to another ext4 file system I would solve the problem using the fantastic tool debugfs. Specifically I can make a list of (filename, crtime) pairs on the source fs (file system), then use debugfs -w on the target fs to read a script with lines of the form

  set_inode_field filename crtime <value>

I've tested this and it works just fine.

• But my target fs is not ext4 but ZFS and although debugfs runs on the target machine, it is entirely useless there. It doesn't even recognize the fs. Another debug tool that lets you alter timestamps by editing an inode directly is fsdb; it too runs on the target machine, but again I can't seem to get it to recognize a ZFS file system.

• I'm told by the folks who sold me the NAS box that debugfs and fsdb are not meant for ZFS filesystems, but they haven't been able to come up with an equivalent. So, after much googling and trying out things I finally decided to post a question here today, hoping someone might have the answer.

I'm surprised at how hard this is turning out to be. The question of how to replicate a dataset so all timestamps are identical seems quite natural from an archival point of view.

Answer 1

Indeed, neither fsdb nor debugfs are likely to be suitable for use with ZFS. What you might need to do instead is find an archive format that will the preserve crtime field that presumably is already set for the files on your fileserver. If there is a version of pax or another archiving tool for your system it may be able to do this (cf. the -pe "preserve everything" flag for pax which it seems in current versions does not preserve "everything" - viz. it does not preserve crtime/birth_time). You will likely have more success finding an archiving application that is "crtime aware" than trying set the creation times by hacking on the ZFS based FreeBSD system with what are likely to be rudimentary tools.

You may be able to find more advanced tools on OpenSolaris based systems like Illumos or SmartOS (e.g. mdb). Whether it would be possible to transfer your data to a ZFS dataset on one of those platforms and then, combining the tools they have with, say, dtrace in order to rewrite the crtime fields is more of a theoretical question. If it worked then you could export the pool and its datasets to FreeBSD - exporting a pool does seem to preserve the crtime time stamps. If you are able to preserve crtime while dumping your ext4 filesystem to a ZFSonLinux dataset on the same host (nb: I have not tested this) you could then use zfs send to transfer the whole filesystem to your NAS.

This core utils bug report may shed some light on the state of user and operating system level tools on Linux. Arguably the filesystem level crtime field of an inode should be difficult to change. While ZFS on FreeBSD "supports" crtime, the state of low level filesystem debugging tools on FreeBSD might not have kept pace in earlier releases (c.f. the zdb manual page). Are you sure you want to "set" (or reset) inode creation times? Or do you want to preserve them after they have been set on a system that already supports them?

On a FreeBSD system if you stat a file stored on a ZFS dataset you will often notice that the crtime field of the file is set to the same time as the ctime field. This is likely because the application that wrote the file did not have access to library and kernel functions required to set crtime at the time the file was "born" and its inode entries were created. There are examples of applications / libraries that try to preserve crtime at the application level such as libarchive(3) (see also: archive_entry_atime(3)) and gracefully handle inode creation if the archive is restored on a filesystem that does not support the crtime field. But that might not be relevant in your case.

As you might imagine, there are a lot of applications that write files to filesystems ... especially with Unix/POSIX systems where "everything is a file". I'm not sure if older applications would need to be modified or recompiled to support those fields, or whether they would pick them up transparently from the host system's C libraries. Applications being used on older FreeBSD releases or on a Linux system without ext4 could be made to run in compatibility mode on an up to date OS, for example, but whether they would properly handle the time fields is a good question.

For me running this little script as sh birthtime_test confirms that file creation times are "turned on" on my FreeBSD systems (all of which use ZFS post v28 i.e. with feature flags):

#!/bin/sh
#birthtime_test
uname -r 
if [ -f new_born ] ; then rm -f new_born ; fi

touch new_born 
sleep 3 
touch -a new_born
sleep 3 
echo "Hello from new_born at:" >> new_born 
echo `date` >> new_born
sleep 3 
chmod o+w new_born

stat -f "Name:%t%N
Born:%t%SB
Access:%t%Sa
Modify:%t%Sm 
Change:%t%Sc" new_born

cat new_born

Output:

9.2-RELEASE-p10
Name:   new_born
Born:   May  7 12:38:35 2015
Access: May  7 12:38:38 2015
Modify: May  7 12:38:41 2015 
Change: May  7 12:38:44 2015
Hello from new_born at:
Thu May 7 12:38:41 EDT 2015

(NB: the chmod operation "changes" but does not "modify" the file contents - this is what the echo command does by adding content to the file. See the touch manual page for explanations of the -m and -a flags).

This is the oldest FreeBSD release I have access to right now. I'd be curious to know how far back in the release cycle FreeBSD is able handle this (on ZFS or UFS2 file systems). I'm pretty sure this has been a feature for quite a while now. There are also OSX and Linux versions of ZFS that it would be useful to know about regarding this feature.

Just one more thing ...

Here is an especially nice feature for simple "forensics". Say we want to send our new_born file back to when time began, back to the leap second that never happened and when - in a moment of timeless time - Unix was born ... :-) ¹. We can just change the date using touch -d and everyone will think new_born is old and wise, right?

Nope:

~/ % touch -d "1970-01-01T00:00:01" new_born                
~/ % stat -f "Name:%t%N   
Born:%t%SB
Access:%t%Sa
Modify:%t%Sm
Change:%t%Sc" new_born
Name:   new_born
Born:   May  7 12:38:35 2015
Access: Jan  1 00:00:01 1970
Modify: Jan  1 00:00:01 1970 
Change: May  7 13:29:37 2015

It's always more truthful to actually be as young as you look :-)

Time and Unix - a subject both practical and poetic: after all, what is "change"; and what does it mean to "modify" or "create" something? Thanks for your great post Silvio - I hope it lives on and gathers useful answers.

---

You can improve and generalize your question if you can be more specific about your requirements for preserving, setting, archiving of file timestamp fields. Don't get me wrong: this is a very good question and it will continue to get up votes for a long time.

You might take a look at Dylan Leigh's presentation Forensic Timestamp Analysis of ZFS or even contact Dylan for clues on how to access crftime information.

---

[1] There was a legend that claimed in the beginning, seconds since long (SSL) ago was never less than date -u -j -f "%Y-%m-%d:%T" "1970-01-01:00:00:01" "+%s" because of a leap second ...