Django return http early flush (chunked response)

Question

I have an algorithm which waits for almost 5 seconds to generate response and I want to send an ack (http 200) to user as soon as he sends request to tell him that his request has been received and wait for 5 seconds.

Generator function:

def chunked_res():
    yield "Chunk 1"

    stop = time.time() + 5    # wait for 5 seconds
    while time.time() < stop:
        pass

    yield "Chunk 2"

And in some view:

response = HttpResponse ( chunked_res() )
response['Connection'] = 'close'
response['Transfer-Encoding'] = 'chunked'
response['status'] = 200
return response

Response in browser:

"Transfer-Encoding: chunked\nstatus: 200\nConnection: close\nContent-Type: text/html; charset=utf-8\n\nChunk 1Chunk 2"

Problem: I am getting required response(Chunk 1, Chunk 2) but after 5 seconds. I want to send "Chunk 1" first and then "Chunk 2" after 5 seconds(update respose). Are there any particular settings/changes to implement this?

Update:

Django = 1.4 python = 2.7

Answer 1

Actually the solution was to make first chunk size of at least 1024 character for browser to show incrementally.

How to stream an HttpResponse with Django

def chunked_res():
    yield "Chunk 1"
    yield " " * 1024  # Encourage browser to render incrementally (either 1024 or 1024-7{length of "chunk 1"} = 1017)

    time.sleep(5)  # wait for 5 seconds

    yield "Chunk 2"


def myview(request):
    g = chunked_res()
    return HttpResponse(g)

If you are using nginx then you have to set proxy_buffering=off, for server to flush response as 1024 data chunk ready. http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size

Works with: HTTP/1.0 and HTTP/1.1

Answer 2

You really want to avoid doing this...

stop = time.time() + 5    # wait for 5 seconds
while time.time() < stop:
    pass

It will cause the CPU usage of your process to spike to 100% for 5 seconds, which is a huge amount for a web application. In a shared hosting environment you risk getting a nasty email from your hosting provider, and if you've got your own server, those cycles you're eating up in the loop could be usefully deployed in handling other requests. Do this instead:

time.sleep(5)

That will put the thread (or process) to sleep for 5 seconds, and the kernel is then free to schedule some other task, or sleep the CPU (saving power and cooling costs) if there's nothing else to do.