How can I serve unbuffered CGI content from Apache 2?

Question

I would like to be able to allow a user to view the output of a long-running GCI script as it is generated rather than after the script is complete. However even when I explicitly flush STDOUT the server seems to wait for the script to complete before sending the response to the client. This is on a Linux server running Apache 2.2.9.

Example python CGI:

#!/usr/bin/python

import time
import sys


print "Content-type: text/plain"
print
for i in range(1, 10):
        print i
        sys.stdout.flush()
        time.sleep(1)

print "Done."

Similar example in perl:

#!/usr/bin/perl

print "Content-type: text/plain\n\n";

for ($i = 1; $i <= 10 ; $i++) {
        print "$i\n";
        sleep(1);
}

print "Done.";

This link says as of Apache 1.3 CGI output should be unbuffered (but this might apply only to Apache 1.x): http://httpd.apache.org/docs/1.3/misc/FAQ-F.html#nph-scripts

Any ideas?

Answer 1

Randal Schwartz's article Watching long processes through CGI explains a different (and IMHO, better) way of watching a long running process.

Answer 2

Flushing STDOUT can help. For example, the following Perl program should work as intended:

#!/usr/bin/perl

use strict;
use warnings;

local $| = 1;

print "Content-type: text/plain\n\n";

for ( my $i = 1 ; $i <= 10 ; $i++ ) {
    print "$i\n";
    sleep(1);
}

print "Done.";

Answer 3

You must put your push script into a special directory wich contain a special .htaccess with this environnment specs:

Options +ExecCGI
AddHandler cgi-script .cgi .sh .pl .py
SetEnvIfNoCase Content-Type \
"^multipart/form-data;" "MODSEC_NOPOSTBUFFERING=Do not buffer file uploads"
SetEnv no-gzip dont-vary

Answer 4

According to CGI::Push,

Apache web server from version 1.3b2 on does not need server push scripts installed as NPH scripts: the -nph parameter to do_push() may be set to a false value to disable the extra headers needed by an NPH script.

You just have to find do_push equivalent in python.

Edit: Take a look at CherryPy: Streaming the response body.

When you set the config entry "response.stream" to True (and use "yield") CherryPy manages the conversation between the HTTP server and your code like this:

_{(source: cherrypy.org)}