Flask File upload - Getting only partial file

Question

I am trying to upload a CSV file of 60 mb to my application. But when i save the file to server , it only contains partial(25 mb) of total records from the original. This issue is occurring only randomly.

@fileupload.route('/loadfile',methods = ['POST'])
def store_to_db():
for _file in request.files.getlist('filelist'):
        name, ext = FileUtil.get_filename_without_ext(_file.filename)#fetching file name and extension
        fd, path = tempfile.mkstemp(suffix=ext, prefix=name)#saving file as temp
        os.write(fd, _file.read())
        try:
            file_upload_service.savefiletodb(path)
        except Exception as e:
            logger.info(str(e))
        os.close(fd)

Uploader side code:

    data= {}
    url_path = 'localhost:5000/loadfile'
    data['filelist']=open('employe.txt', 'rb')
    resp = requests.post(url=url_path, files=data)

Any idea why its behaving like this ?

Note:

• When i upload through postman, i am not facing any issue.

• If I upload through code , sometimes I get partial file.

• file object in log , when upload via postman

  'employee.TXT' ('text/plain')>

• file object in log , when upload via code

  'employee.TXT' ('None')>

Answer 1

If the issue is with the file size which is exceeding the file size limit, Flask will raise a RequestEntityTooLarge exception.

If this is the case, you can simply configure the MAX_CONTENT_LENGTH config key. For example,

from flask import Flask, Request

app = Flask(__name__)
app.config['MAX_CONTENT_LENGTH'] = 20 * 1024 * 1024

The above code will set the File upload limit to 20 megabytes. If the user tries to upload a file larger than the set limit (in this case 20 MB), Then Flask will raise a RequestEntityTooLarge exception.

And then if required, Allowed file extensions can also be specified by,

ALLOWED_EXTENSIONS = {'txt', 'docx','pdf', 'png', 'jpg', 'jpeg', 'gif'}

For more help, Refer the Flask Docs for Uploading files.

Answer 2

1. try to set app.config['MAX_CONTENT_LENGTH'] here

2. try to set headers like return Response(mimetype='text/plain') here is reference for the same

3. try this tutorial