How do file descriptors work?

Question

Can someone tell me why this does not work? I'm playing around with file descriptors, but feel a little lost.

#!/bin/bash echo "This" echo "is" >&2 echo "a" >&3 echo "test." >&4 

The first three lines run fine, but the last two error out. Why?

Answer 1

File descriptors 0, 1 and 2 are for stdin, stdout and stderr respectively.

File descriptors 3, 4, .. 9 are for additional files. In order to use them, you need to open them first. For example:

exec 3<> /tmp/foo  #open fd 3. echo "test" >&3 exec 3>&- #close fd 3. 

For more information take a look at Advanced Bash-Scripting Guide: Chapter 20. I/O Redirection.

Answer 2

It's an old question but one thing needs clarification.

While the answers by Carl Norum and dogbane are correct, the assumption is to change your script to make it work.

What I'd like to point out is that you don't need to change the script:

#!/bin/bash echo "This" echo "is" >&2 echo "a" >&3 echo "test." >&4 

It works if you invoke it differently:

./fdtest 3>&1 4>&1 

which means to redirect file descriptors 3 and 4 to 1 (which is standard output).

The point is that the script is perfectly fine in wanting to write to descriptors other than just 1 and 2 (stdout and stderr) if those descriptors are provided by the parent process.

Your example is actually quite interesting because this script can write to 4 different files:

./fdtest >file1.txt 2>file2.txt 3>file3.txt 4>file4.txt 

Now you have the output in 4 separate files:

$ for f in file*; do echo $f:; cat $f; done file1.txt: This file2.txt: is file3.txt: a file4.txt: test. 

What is more interesting about it is that your program doesn't have to have write permissions for those files, because it doesn't actually open them.

For example, when I run sudo -s to change user to root, create a directory as root, and try to run the following command as my regular user (rsp in my case) like this:

# su rsp -c '../fdtest >file1.txt 2>file2.txt 3>file3.txt 4>file4.txt' 

I get an error:

bash: file1.txt: Permission denied 

But if I do the redirection outside of su:

# su rsp -c '../fdtest' >file1.txt 2>file2.txt 3>file3.txt 4>file4.txt 

(note the difference in single quotes) it works and I get:

# ls -alp total 56 drwxr-xr-x 2 root root 4096 Jun 23 15:05 ./ drwxrwxr-x 3 rsp  rsp  4096 Jun 23 15:01 ../ -rw-r--r-- 1 root root    5 Jun 23 15:05 file1.txt -rw-r--r-- 1 root root   39 Jun 23 15:05 file2.txt -rw-r--r-- 1 root root    2 Jun 23 15:05 file3.txt -rw-r--r-- 1 root root    6 Jun 23 15:05 file4.txt 

which are 4 files owned by root in a directory owned by root - even though the script didn't have permissions to create those files.

Another example would be using chroot jail or a container and run a program inside where it wouldn't have access to those files even if it was run as root and still redirect those descriptors externally where you need, without actually giving access to the entire file system or anything else to this script.

The point is that you have discovered a very interesting and useful mechanism. You don't have to open all the files inside of your script as was suggested in other answers. Sometimes it is useful to redirect them during the script invocation.

To sum it up, this:

echo "This" 

is actually equivalent to:

echo "This" >&1 

and running the program as:

./program >file.txt 

is the same as:

./program 1>file.txt 

The number 1 is just a default number and it is stdout.

But even this program:

#!/bin/bash echo "This" 

can produce a "Bad descriptor" error. How? When run as:

./fdtest2 >&- 

The output will be:

./fdtest2: line 2: echo: write error: Bad file descriptor 

Adding >&- (which is the same as 1>&-) means closing the standard output. Adding 2>&- would mean closing the stderr.

You can even do a more complicated thing. Your original script:

#!/bin/bash echo "This" echo "is" >&2 echo "a" >&3 echo "test." >&4 

when run with just:

./fdtest 

prints:

This is ./fdtest: line 4: 3: Bad file descriptor ./fdtest: line 5: 4: Bad file descriptor 

But you can make descriptors 3 and 4 work, but number 1 fail by running:

./fdtest 3>&1 4>&1 1>&- 

It outputs:

./fdtest: line 2: echo: write error: Bad file descriptor is a test. 

If you want descriptors both 1 and 2 fail, run it like this:

./fdtest 3>&1 4>&1 1>&- 2>&- 

You get:

a test. 

Why? Didn't anything fail? It did but with no stderr (file descriptor number 2) you didn't see the error messages!

I think it's very useful to experiment this way to get a feeling of how the descriptors and their redirection work.

Your script is a very interesting example indeed - and I argue that it is not broken at all, you were just using it wrong! :)