Werkzeug response too slow

Question

I have the following Werkzeug application for returning a file to the client:

from werkzeug.wrappers import Request, Response

@Request.application
def application(request):    
    fileObj = file(r'C:\test.pdf','rb')
    response = Response( response=fileObj.read() )
    response.headers['content-type'] = 'application/pdf'
    return response

The part I want to focus on is this one:

response = Response( response=fileObj.read() )

In this case the response takes about 500 ms (C:\test.pdf is a 4 MB file. Web server is in my local machine).

But if I rewrite that line to this:

response = Response()
response.response = fileObj

Now the response takes about 1500 ms. (3 times slower)

And if write it like this:

response = Response()
response.response = fileObj.read()

Now the response takes about 80 seconds (that's right, 80 SECONDS).

Why is there that much difference between the 3 methods?
And why is the third method sooooo slow?

Answer 1

The answer to that is pretty simple:

• x.read() <- reads the whole file into memory, inefficient
• setting response to a file: very inefficient as the protocol for that object is an iterator. So you will send the file line by line. If it's binary you will send it with random chunk sizes even.
• setting response to a string: bad idea. It's an iterator as mentioned before, so you are now sending each character in the string as a separate packet.

The correct solution is to wrap the file in the file wrapper provided by the WSGI server:

from werkzeug.wsgi import wrap_file
return Response(wrap_file(environ, yourfile), direct_passthrough=True)

The direct_passthrough flag is required so that the response object does not attempt to iterate over the file wrapper but leaves it untouched for the WSGI server.