Java hangs even though script's execution is completed

Question

I'm trying to execute a script from within my java code which looks like:

Process p = Runtime.getRuntime().exec(cmdarray, envp, dir); // cmdarray is a String array
// consisting details of the script and its arguments

final Thread err = new Thread(...); // Start reading error stream
err.start();
final Thread out = new Thread(...); // Start reading output stream
out.start();
p.waitFor();
// Close resources 

The execution of the script is over(it's pid is no more), but java is stuck on waitFor() method of the process!. And yes, I'm reading output and error streams in 2 separate threads. Yes, they are being joined at the end(after waitFor()).

The script basically installs a few RPMs(like 10 or so) and configures them. So the script runs for a little over 60 seconds.

It looks similar to the following:

#!/bin/sh

#exec 3>&1 >/var/log/some_log 2>&1

# If the above line is uncommented, Java recognizes that the 
# process is over and terminates fine.

tar xzf a-package-having-rpms.tar.gz
cd unpacked-folder
(sh installer-script.sh) #This installs 10 odd rpms in a subshell and configures them
cd ..
rm -rf unpacked-folder

exit 0

Shockingly enough, if I put the following line in the script(at the top), Java understands the script is over and it terminates the process perfectly.

exec 3>&1 > /var/log/some_log 2>&1

For the record, the script doesn't generate any output. Zero chars!. So putting exec statement there makes no sense!

But still, magically enough, putting exec statement in the script makes java work!. Why??

How can I avoid that illogical exec statement in the script?.

If you are interested in what installer-script.sh looks like then:

#!/bin/sh

exec 3>&1 >>/var/log/another-log.log 2>&1
INSDIR=$PWD
RPMSDIR=$INSDIR/RPMS
cd $RPMSDIR
#
rpm -i java-3.7.5-1.x86_64.rpm
rpm -i --force perl-3.7.5-1.x86_64.rpm
rpm -i --nodeps mysql-libs-5.0.51a-1.vs.i386.rpm
rpm -i --nodeps mysql-5.0.51a-1.vs.i386.rpm
rpm -i --nodeps mysql-server-5.0.51a-1.vs.i386.rpm
rpm -i --nodeps perl-DBD-MySQL-3.0007-2.el5.i386.rpm
rpm -i --nodeps perl-XML-Parser-2.34-6.1.2.2.1.i386.rpm
.
.
.

Now, why exec command in the first script is required for Java to know that the process is over? How can I avoid that exec?, esp. since the first script doesn't produce any output.

Waiting for answers with bated breath!

Answer 1

My guess is that Java doesn't think the script is over until the pipes it passed to it via stdin/stdout/stderr are closed by the sub-process. That is, there are no more active reader processes on stdin, no more active writer processes on stdout/stderr.

When you're reading on a pipe, you don't receive an end-of-file indication until there are no more processes that have the pipe open for output. So if a process forks and the new process inherits an open file handle, then the original process terminates, there's still a process with the file open, and a reader will still wait.

Similarly with a pipe you're writing, you won't receive a "broken pipe" signal until the last reader closes the pipe.

This problem generally arises when your script forks off background tasks (like newly-installed services) which inherit stdin/stdout/stderr.

By using exec, you're explicitly breaking the inheritance chain of these pipes so that the background processes don't use them.

If on Linux, check /proc/*/fd for any new services and see if their stdin/stdout/stderr is the same pipe that your java process passes to your script.

The same situation often happens when you run the /etc/init.d/xxx scripts: they complete normally when you run them from the command-line but seem to hang when you run them from some kind of monitor.

EDIT:

You say that the installer script contains the line:

exec 3>&1 >>/var/log/another-log.log 2>&1

The first term, 3>&1, clones stdout to file-descriptor 3 (see Redirections in man bash). As far as I know, fd 3 has no special meaning. Then it replaces stdout by opening /var/log/another-log.log and replaces stderr by cloning stdout. See the Redirections section of the bash man page

This means the the new file-descriptor 3 is open for writing on the pipe that was originally passed in as STDOUT. Programs that expect to be system service daemons will often close file descriptors 0 (STDIN), 1 (STDOUT) and 2 (STDERR) but may not bother with any others. Also, now that the shell has opened FD-3 it will pass that open file to any command it executes, including background commands.

Do you know if there's any particular reason that the installer opens FD 3? My guess is that if you simply remove the "3>&1" term from the installer your problem will be solved. This will allow you to remove the exec from your script entirely