Safer alternative to MATLAB's `system` command

Question

I have been using MATLAB's system command to get the result of some linux commands, like in the following simple example:

[junk, result] = system('find ~/ -type f')

This works as expected, unless the user types into MATLAB's command window at the same time. Which during a long find command is not uncommon. If this happens then the user's input seems to get mixed up with the result of the find command (and then things break).

As an example, instead of:

/path/to/file/one
/path/to/file/two
/path/to/file/three
/path/to/file/four

I might get:

J/path/to/file/one
u/path/to/file/two
n/path/to/file/three
k/path/to/file/four

In order to demonstrate this easily, we can run something like:

[junk, result] = system('cat')

Type something into the command window and press CTRL+D to close the stream. The result variable will be whatever you typed in to the command window.

Is there a safer way for me to call system commands from MATLAB without risking corrupted input?

Answer 1

Wow. That behavior is surprising. Sounds worth reporting as a bug to MathWorks. I tested it on OS X, and see the same behavior.

As a workaround, you could re-implement system() using calls to the java.lang.Process and related objects in the JVM embedded in Matlab.

You'll need to:

• Use polling to keep Matlab's own input, especially Ctrl-C, live
• Use shell processing instead of passing the commands directly to ProcessBuilder to support expansion of ~ and other variables and wildcards, and to support specifying commands and their arguments in a single string like Matlab's system does. ** Alternately, you could expose the arguments-array form if you want lower-level control and don't want to deal with escaping and quoting strings for the shell. Both would be useful.
• Redirect the output to files, or periodically drain the child process's output buffers in your polling code.

Here's an example.

function [status,out,errout] = systemwithjava(cmd)
%SYSTEMCMD Version of system implemented with java.lang features
%
% [status,out,errout] = systemwithcmd(cmd)
%
% Written to work around issue with Matlab UI entry getting mixed up with 
% output captured by system().

if isunix
    % Use 'sh -s' to enable processing of single line command and expansion of ~
    % and other special characters, like the Matlab system() does
    pb = java.lang.ProcessBuilder({'bash', '-s'});
    % Redirect stdout to avoid filling up buffers
    myTempname = tempname;
    stdoutFile = [myTempname '.systemwithjava.out'];
    stderrFile = [myTempname '.systemwithjava.err'];
    pb.redirectOutput(java.io.File(stdoutFile));
    pb.redirectError(java.io.File(stderrFile));
    p = pb.start();
    RAII.cleanUpProcess = onCleanup(@() p.destroy());
    RAII.stdoutFile = onCleanup(@() delete(stdoutFile));
    RAII.stderrFile = onCleanup(@() delete(stderrFile));
    childStdin = java.io.PrintStream(p.getOutputStream());
    childStdin.println(cmd);
    childStdin.close();
else
    % TODO: Fill in Windows implementation here    
end

% Poll instead of waitFor() so Ctrl-C stays live
% This try/catch mechanism is lousy, but there is no isFinished() method.
% Could be done more cleanly with a Java worker that did waitFor() on a
% separate thread, and have the GUI event thread interrupt it on Ctrl-C.
status = [];
while true
    try
        status = p.exitValue();
        % If that returned, it means the process is finished
        break;
    catch err
        if isequal(err.identifier, 'MATLAB:Java:GenericException') ...
                && isa(err.ExceptionObject, 'java.lang.IllegalThreadStateException')
            % Means child process is still running
            % (Seriously, java.lang.Process, no "bool isFinished()"?
            % Just continue
        else
            rethrow(err);
        end
    end
    % Pause to allow UI event processing, including Ctrl-C
    pause(.01);
end

% Collect output
out = slurpfile(stdoutFile);
errout = slurpfile(stderrFile);
end

function out = slurpfile(file)
fid = fopen(file, 'r');
RAII.fid = onCleanup(@() fclose(fid));
out = fread(fid, 'char=>char')'; %'
end

I tried this out as best as I could, and it looks like it keeps the child process's output separate from keyboard input to the Matlab IDE. Keyboard input is buffered up and executed as additional commands after systemwithjava() returns. Ctrl-C stays live and will interrupt the function, letting the child process get killed.

Answer 2

Thanks goes to Andrew Janke for helping me find this solution.

To easily reproduce the error we can run the command:

[ret, out] = system('sleep 2');

If we type some characters while this is running, the out variable will be contaminated with what we typed.

The solution to this problem is to redirect stdin from /dev/null like the following:

[ret, out] = system('sleep 2 < /dev/null');

This stops the out variable from being contaminated by user input.

Interestingly though this seems to fix the original test-case for the current MATLAB session (tested on R2014a OSX and R2013b Linux) so if we do another [ret, out] = system('sleep 2'); the output is no longer contaminated by user input.