PHP: trying to get fgets() to trigger both on CRLF, CR and LF

Question

I'm reading streams in PHP, using proc_open and fgets($stdout), trying to get every line as it comes in.

Many linux programs (package managers, wget, rsync) just use a CR (carriage return) character for lines which periodically updates "in place", like download progress. I'd like to catch these updates (as separate lines) as soon as they happen.

At the moment, fgets($stdout) just keeps reading until a LF, so when progress is going very slowly (big file for example) it just keeps on reading until it's completely done, before returning all the updated lines as one long string, including the CRs.

I've tried setting the "mac" option to detect CRs as line endings:

ini_set('auto_detect_line_endings',true); 

But that doesn't seem to work.

Now, stream_get_line would allow me to set CRs as line breaks, but not a "catch all" solution which treats both CRLF, CR and LF as delimiters.

I could of course read the whole line, split it using various PHP methods and replace all types of linebreaks with LFs, but it's a stream, and I want PHP to be able to get an indication of progress while it's still running.

So my question:

How can I read from the STDOUT pipe (from proc_open) until a LF or CR happens, without having to wait until the whole line is in?

Thanks in advance!

Solution:

I used Fleshgrinder's filter class to replace \r with \n in the stream (see accepted answer), and replaced fgets() with fgetc() to get more "realtime" access to contents of STDOUT:

$stdout = $proc->pipe(1);
stream_filter_register("EOL", "EOLStreamFilter");
stream_filter_append($stdout, "EOL"); 

while (($o = fgetc($stdout))!== false){
    $out .= $o;                            // buffer the characters into line, until \n.
    if ($o == "\n"){echo $out;$out='';}    // can now easily wrap the $out lines in JSON
}

Answer 1

Use a stream filter to normalize your new line characters before consuming the stream. I created the following code that should do the trick based on the example from PHP’s manual page on stream_filter_register.

Code is untested!

<?php

// https://php.net/php-user-filter
final class EOLStreamFilter extends php_user_filter {

    public function filter($in, $out, &$consumed, $closing)
    {
        while ($bucket = stream_bucket_make_writeable($in)) {
            $bucket->data = str_replace([ "\r\n", "\r" ], "\n", $bucket->data);
            $consumed += $bucket->datalen;
            stream_bucket_append($out, $bucket);
        }
        return PSFS_PASS_ON;
    }

}

stream_filter_register("EOL", "EOLStreamFilter");

// Open stream …

stream_filter_append($yourStreamHandle, "EOL");

// Perform your work with normalized EOLs …

---

EDIT: The comment Mark Baker posted on your question is true. Most Linux distributions are using a line buffer for STDOUT and it is possible that Apple is doing the same. On the other hand most STDERR streams are unbuffered. You could try to redirect the output of the program to another pipe (e.g. STDERR or any other) and see if you have more luck with that.