xargs: tar: terminated by signal 13

Question

I'm running the following command to copy files from a list given by git to an other directory using tar to keep the rights.

git ls-files -z | xargs -0 tar -c | tar -x -C ~/tmp

This seems to work on some repos, but not on mine:

xargs: tar: terminated by signal 13

Answer 1

Answering my own question¹:

Signal 13 is broken pipe: the receiving end stopped reading but we're still piping into the pipe.

My first hint was a problematic file, so let's add the -t option to xargs so it prints the command:

git ls-files -z | xargs -t -0 tar -c | tar -x -C ~/tmp

Output:

tar -c [long list of files (2994 files)]
tar -c [sightly less long list of files (~700 files)]

At this point the problem appears clear: we're piping two tar calls into one, so the pipe is broken (signal 13).

Indeed, reading the xargs manual, we can read that:

The command line for command is built up until it reaches a system-defined limit (unless the -n and -L options are used). The specified command will be invoked as many times as necessary to use up the list of input items.

You can check out xargs's limits with xargs --show-limits. xargs generates multiple commands if the arguments length exceeds the systems limits.

As the system limit for the command line is high (at least on my system, it's 131072 bytes which with my files equivalents to ~3000 files), this can be defined as the general case. This means that if the file list fits within the system-defined limit, the initial command works well.

We can reproduce for every case (well, with at least 2 files), by limiting the number of files xargs will throw in every call with the -L option:

git ls-files -z | xargs -L 1 -t -0 tar -c | tar -x -C ~/tmp

The solution is actually to drop xargs altogether, by using tar's -T option, to read the list of files from a file (or -, meaning stdin):

git ls-files -z | tar --null -T - -c | tar -x -C ~/tmp

or

git ls-files | tar -T - -c | tar -x -C ~/tmp

---

¹: https://discuss.circleci.com/t/local-checkout-fails-using-cli-tar-terminates-with-signal-13/28632/14