Shell redirection i/o order

Question

I'm playing with i/o shell redirection. The commands I've tried (in bash):

ls -al *.xyz 2>&1 1> files.lst 

and

ls -al *.xyz 1> files.lst 2>&1 

There is no any *.xyz file in current folder.

These commands gives me the different results. The first command shows an error message ls: *.xyz: No such file or directory on the screen. But the second one prints this error message to the file. Why did the first command failed to write an err output to the file?

Answer 1

The Bash manual has a clear example (similar to yours) to show that the order matters and also explains the difference. Here's the relevant part excerpted (emphasis mine):

Note that the order of redirections is significant. For example, the command

ls > dirlist 2>&1

directs both standard output (file descriptor 1) and standard error (file descriptor 2) to the file dirlist, while the command

ls 2>&1 > dirlist

directs only the standard output to file dirlist, because the standard error was made a copy of the standard output before the standard output was redirected to dirlist.

This post explains it from the POSIX viewpoint.

Confusions happen due to a key difference. > redirects not by making left operand (stderr) point to right operand (stdout) but by making a copy of the right operand and assigning it to the left. Conceptually, assignment by copy and not by reference.

So reading from left-to-right which is how this is interpreted by Bash: ls > dirlist 2>&1 means redirect stdout to the file dirlist, followed by redirection of stderr to whatever stdout is currently (which is already the file dirlist). However, ls 2>&1 > dirlist would redirect stderr to whatever stdout is currently (which is the screen/terminal) and then redirect stdout to dirlist.

Answer 2

Redirections are:

• processed from left to right.
• interpreted iteratively:
  • an earlier redirection can affect a later one:
    • if an earlier redirection has redirected a given stream (identified by a file descriptor number, such as 1 for stdout (the default), and 2 for stderr), later redirections targeting that stream refer to the already-redirected version.
  • but not vice versa - a later redirection has no retroactive effect on the the target of an earlier redirection:
    • e.g., if you specify file descriptor 1 as the target in an earlier redirection, what 1 means at that time is locked in, even if 1 is redirected later.
• Note, however, that output isn't actually sent until all redirections are in place, and that any redirection-target output files are created or truncated before command execution begins (this is the reason why you can't read from and redirect output to the same file with a single command).

Applied to the example from the question:

• >file 2>&1:

  • >file first redirects stdout (file descriptor 1, implied by not prefixing > with a file descriptor number) to output file file
  • 2>&1 then redirects stderr (2) to the already redirected stdout (1).
  • The net effect is that both original streams end up in file.

• 2>&1 >file:

  • 2>&1 first redirects stderr to the then-original stdout; since file descriptor 2 participates in no further redirections, stderr output will therefore go to whatever stdout was defined as at that point - i.e., the original stdout, given that this is the first redirection.
    • Technically, the original stdout file descriptor is duplicated, and that duplicate is what stderr then refers to, which explains why it isn't affected by a later redirection of stdout.
  • >file then redirects the original stdout to file - but that has no effect anymore on the already locked-in redirection of stderr.
  • The net effect is that only sent-directly-to-stdout output is captured in file, while sent-to-stderr output is output to (the original, unredirected) stdout.