Addressing sys.excepthook error in bash script

Question

I've written a bash script that is doing exactly what I want it to do, but kicking out the following error:

close failed in file object destructor: sys.excepthook is missing lost sys.stderr

I'm completely stumped on how to address this. Here is the script:

#!/bin/bash

usage () { echo "${0##*/} inputfile outputfile"; exit 1; }

(($#==2)) || usage

INPUTFILE="$1"
OUTPUTFILE="$2"

#  All that is written between between the 'cat' command and 
#+ 'EOF' will be sent to the output file.
cat <<EOF >$OUTPUTFILE
$(date "+Generated on %m/%d/%y at %H:%M:%S")

DATA AUDIT: $1

------------
COLUMN NAMES
------------

$(csvcut -n $INPUTFILE)

---------------------------------------
FIRST TEN ROWS OF FIRST FIVE COLUMNS 
---------------------------------------

$(csvcut -c 1,2,3,4,5 $INPUTFILE | head -n 10)

------------
COLUMN STATS
------------

$(csvcut $INPUTFILE | csvstat )

---END AUDIT
EOF

echo "Audited!"

I am pretty new to shell scripts and very new to python. I would be grateful for any help.

Answer 1

I was seeing this error when piping output from a Python 2.6.2 script into the head command in bash on Ubuntu 9.04. I added try blocks to close stdout and stderr before exiting the script:

try:
    sys.stdout.close()
except:
    pass
try:
    sys.stderr.close()
except:
    pass

I am no longer seeing the error.

Answer 2

There are two steps you need to perform:

Step 1:

In your csvcut script, find all locations where sys.stdout.write() are called, make sure sys.stdout.flush() is called after each write().

Step 2:

With step 1 completed you should now be able to capture IOError within the Python script. Below is one example on how to handle broken pipe:

try:
    function_with_sys_stdout_write_call()
except IOError as e:
    # one example is broken pipe
    if e.strerror.lower() == 'broken pipe':
        exit(0)
    raise       # other real IOError

Hope it helps!

Answer 3

I assume that the csvcut Python script is otherwise functional but is throwing up an error when it tries to close files and exit.

If, as you say, the script is otherwise working and assuming that the error 'csvcut' is throwing up outputs to stderr then redirecting it to /dev/null would be a temp fix.

cat <<EOF >$OUTPUTFILE 2>/dev/null

Naturally any other error messages in your heredoc will also redirect there.