Menu
I have no choice but to retrieve some external data by means of several Runtime.exec() calls to a VBScript. I truly hate this implementation, as I lose my cross-platform flexibility, but I may eventually develop similar *nix scripts to at least mitigate the problem. Before anyone asks, I cannot work around the need to call an external script to gather my data. I will live with the problems that causes.
The exec() processes are run in a custom class that extends Runnable. It uses a BufferedReader to read in the data from getInputStream().
Edit: more code added as requested, but I don't see how the extra code is relevant :) I hope it helps, because it took a while to format! Oh, and go easy on my code style if it's ugly, but constructive criticism is encouraged...
public class X extends JFrame implements Runnable {
...
static final int THREADS_MAX = 4;
ExecutorService exec;
...
public static void main(String[] args) {
...
SwingUtilities.invokeLater(new X("X"));
} // End main(String[])
public X (String title) {
...
exec = Executors.newFixedThreadPool(THREADS_MAX);
...
// Create all needed instances of Y
for (int i = 0; i < objects.length; i++) {
Y[i] = new Y(i);
} // End for(i)
// Initialization moved here for easy single-thread testing
// Undesired, of course
for (int i = 0; i < objects.length; i++) {
Y[i].initialize(parent);
} // End for(i)
} // End X
class Y implements Runnable {
// Define variables/arrays used to capture data here
String computerName = "";
...
public Y(int rowIndex) {
row = rowIndex;
...
computerName = (String)JTable.getValueAt(row, 0);
...
exec.execute(this);
} // End Y(int)
public void run() {
// Initialize variables/arrays used to capture data here
...
// Initialization should be done here for proper threading
//initialize(parent);
} // End run()
public void initialize(Z obj) {
runTime = Runtime.getRuntime();
...
try {
process = runTime.exec("cscript.exe query.vbs " + computerName);
stdErr = process.getErrorStream();
stdIn = process.getInputStream();
isrErr = new InputStreamReader(stdErr);
isrIn = new InputStreamReader(stdIn);
brErr = new BufferedReader(isrErr);
brIn = new BufferedReader(isrIn);
while ((line = brIn.readLine()) != null) {
// Capture, parse, and store data here
...
} // End while
} catch (IOException e) {
System.out.println("Unable to run script");
} catch (Exception e) {
e.printStackTrace();
} finally {
try {
stdErr.close();
stdIn. close();
isrErr.close();
isrIn. close();
brErr. close();
brIn. close();
} catch (IOException e) {
System.out.println("Unable to close streams.");
} // End try
} // End try
} // End initialize(Z)
...
} // End class Y
} // End class X
If I execute the commands individually, I gather the data as I expect. However, if I execute the commands in the run() block of the class (meaning the calls are concurrent, as I am hoping for), it appears as if only one input stream is generated, which all BufferedReaders consume concurrently.
To debug the issue, I output each consumed line on the console prefixed by which instance of my class was reading the input stream. I expect something like the following, understanding that they may be out of order from instance-to-instance, but the line order of a single instance would be intact:
exec 0: Line1
exec 1: Line1
exec 2: Line1
exec 0: Line2
exec 1: Line2
exec 2: Line2
exec 0: Line3
exec 1: Line3
exec 2: Line3
...
What's strange is I get the expected number of instances of the very first line of the output (Microsoft (R) Windows Script Host Version 5.7), but after this line, only one process continues to produce data in the input stream, and all readers randomly-consume this one stream, such as this example:
exec 2: Microsoft (R) Windows Script Host Version 5.7
exec 0: Microsoft (R) Windows Script Host Version 5.7
exec 1: Microsoft (R) Windows Script Host Version 5.7
exec 0: line2
exec 1: line3
exec 2: line4
...
To make matters worse, the readers stall and readLine() never returns null. I read that this type of behavior might have something to do with the buffer size, but when I run only two concurrent threads, even with a short output, it still exhibits the same behavior. Nothing is captured in stdErr to indicate there is a problem.
To see if this was a limitation of the script host, I created a batch file that STARTs multiple instances of the script concurrently. I should state this was run outside of Java, in a cmd shell, and launches several of its own shells. However, each concurrent instance fully returned the expected results and behaved well.
Edit: As another troubleshooting idea, I decided to re-enable concurrency, but stagger my initialization method by inserting the following into my Y.run() block:
try {
Thread.sleep((int)(Math.random() * 1200));
} catch (InterruptedException e) {
System.out.println("Can't sleep!");
} // End try
initialize(monitor);
into my code. I begin to see multiple outputs for the first few lines, but it quickly reverts to multiple consumers consuming the same producer, and as soon as the first completed stream closes, the rest of the consumers fire exceptions. The next consumer fires an IOException: Read error, and the rest fire IOException: Stream closed!
According to maaartinus, it IS possible to run multiple, concurrent InputStreams, so now the question becomes what is causing the undesired behavior? How can I independently grab their input streams? I don't want to have to write to a temporary file just to process the data back in, if I can avoid it.
477
I think you need to be careful about the scope of the IO variables. Here is a quick code that works perfectly well, with concurrent Input Streams from 4 child processes...
import java.io.*;
public class MultiExec {
private final static String[] comLines = {
"date",
"ls /var/spool/postfix",
"ls -F /usr/local/bin",
"wc -l /etc/apache2/apache2.conf"};
public void execute() {
for (int i = 0 ; i < comLines.length ; i++) {
ExecutableChild ec = new ExecutableChild (i, comLines[i]);
new Thread (ec).start();
}}
public class ExecutableChild implements Runnable {
private int prIndex;
private String executable;
public ExecutableChild (int k, String cmd) {
prIndex = k;
executable = cmd;
}
public void run () {
try {
Process child = Runtime.getRuntime().exec(executable);
BufferedReader br = new BufferedReader (new InputStreamReader (
child.getInputStream()));
for (String s = br.readLine() ; s != null ; s = br.readLine())
System.out.println ("[" + prIndex + "] " + s);
br.close();
} catch (IOException ioex) {
System.err.println ("IOException for process #"+
prIndex+ ": " + ioex.getMessage());
}}}
public static void main (String[] args) {
new MultiExec().execute();
}
}
Output from the above code (% javac MultiExec.java; java MultiExec)
[2] tomcat*
[0] Thu Jan 20 18:38:31 CST 2011
[3] 368 /etc/apache2/apache2.conf
[1] active
[1] bounce
[1] corrupt
[1] defer
[1] deferred
[1] etc
[1] flush
[1] hold
[1] incoming
[1] lib
[1] maildrop
[1] pid
[1] private
[1] public
[1] saved
[1] trace
[1] usr
[1] var
If you get us the source code for your attempt, we could discuss it. Good wishes, - M.S.
=============================================================================
Edit: DN: I understand your concerns about 1-line outputs. Lets have a small script...
#!/usr/bin/perl -w
foreach (1..50) {
print "$_\n";
}
and an edited version of the above Java Code... comLines have changed, and a Thread.sleep added after every println()
public class MultiExec {
private final static String[] comLines = {
"ls /var/spool/postfix",
"perl count50.pl",
"cat MultiExec.java",
"head -40 /etc/apache2/apache2.conf"};
public void execute() {
for (int i = 0 ; i < comLines.length ; i++) {
ExecutableChild ec = new ExecutableChild (i, comLines[i]);
new Thread (ec).start();
}}
public class ExecutableChild implements Runnable {
private int prIndex;
private String executable;
public ExecutableChild (int k, String cmd) {
prIndex = k;
executable = cmd;
}
public void run () {
try {
Process child = Runtime.getRuntime().exec(executable);
BufferedReader br = new BufferedReader (new InputStreamReader (
child.getInputStream()));
for (String s = br.readLine() ; s != null ; s = br.readLine()) {
System.out.println ("[" + prIndex + "] " + s);
try {
Thread.sleep (20);
} catch (InterruptedException intex) {
}}
br.close();
} catch (IOException ioex) {
System.err.println ("IOException for process #"+
prIndex+ ": " + ioex.getMessage());
}}}
public static void main (String[] args) {
new MultiExec().execute();
}}
Here is the output now (after compile/run) ...
[0] active
[1] 1
[2] import java.io.*;
[3] #
[2]
[0] bounce
[1] 2
[3] # Based upon the NCSA server configuration files originally by Rob McCool.
[2] public class MultiExec {
[1] 3
[0] corrupt
[3] #
[1] 4
[2]
[0] defer
[3] # This is the main Apache server configuration file. It contains the
[2] private final static String[] comLines = {
[0] deferred
[1] 5
[3] # configuration directives that give the server its instructions.
[2] "ls /var/spool/postfix",
[0] etc
[1] 6
[3] # See http://httpd.apache.org/docs/2.2/ for detailed information about
[2] "perl count50.pl",
[0] flush
[1] 7
[3] # the directives.
[2] "cat MultiExec.java",
[1] 8
[0] hold
[3] #
[1] 9
[2] "head -40 /etc/apache2/apache2.conf"};
[0] incoming
[3] # Do NOT simply read the instructions in here without understanding
[2]
[0] lib
[1] 10
[3] # what they do. They're here only as hints or reminders. If you are unsure
[1] 11
[2] public void execute() {
[0] maildrop
[3] # consult the online docs. You have been warned.
[2] for (int i = 0 ; i < comLines.length ; i++) {
[0] pid
[1] 12
[3] #
[1] 13
[2] ExecutableChild ec = new ExecutableChild (i, comLines[i]);
[0] private
[3] # The configuration directives are grouped into three basic sections:
[1] 14
[2] new Thread (ec).start();
[0] public
[3] # 1. Directives that control the operation of the Apache server process as a
[2] }}
[1] 15
[0] saved
[3] # whole (the 'global environment').
[1] 16
[0] trace
[2]
[3] # 2. Directives that define the parameters of the 'main' or 'default' server,
[0] usr
[2] public class ExecutableChild implements Runnable {
[1] 17
[3] # which responds to requests that aren't handled by a virtual host.
[0] var
[2]
[1] 18
[3] # These directives also provide default values for the settings
[1] 19
[2] private int prIndex;
[3] # of all virtual hosts.
[1] 20
[2] private String executable;
[3] # 3. Settings for virtual hosts, which allow Web requests to be sent to
[2]
[1] 21
[3] # different IP addresses or hostnames and have them handled by the
[1] 22
[2] public ExecutableChild (int k, String cmd) {
[3] # same Apache server process.
[1] 23
[2] prIndex = k;
[3] #
[1] 24
[2] executable = cmd;
[3] # Configuration and logfile names: If the filenames you specify for many
[2] }
[1] 25
[3] # of the server's control files begin with "/" (or "drive:/" for Win32), the
[2]
[1] 26
[3] # server will use that explicit path. If the filenames do *not* begin
[1] 27
[2] public void run () {
[3] # with "/", the value of ServerRoot is prepended -- so "/var/log/apache2/foo.log"
[1] 28
[2] try {
[3] # with ServerRoot set to "" will be interpreted by the
[1] 29
[2] Process child = Runtime.getRuntime().exec(executable);
[3] # server as "//var/log/apache2/foo.log".
[1] 30
[2] BufferedReader br = new BufferedReader (new InputStreamReader (
[3] #
[1] 31
[2] child.getInputStream()));
[3]
[1] 32
[2] for (String s = br.readLine() ; s != null ; s = br.readLine()) {
[3] ### Section 1: Global Environment
[1] 33
[2] System.out.println ("[" + prIndex + "] " + s);
[3] #
[1] 34
[2] try {
[3] # The directives in this section affect the overall operation of Apache,
[1] 35
[2] Thread.sleep (20);
[3] # such as the number of concurrent requests it can handle or where it
......
The Input Streams are working just fine, don't think I have a problem here. Sorry about the reply getting so long. Wish you the best, and waiting to see your code, - M.S.
121
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
