Write files to disk with python 3.x

Question

Using BottlePy, I use the following code to upload a file and write it to disk :

upload = request.files.get('upload')
raw = upload.file.read()
filename = upload.filename
with open(filename, 'w') as f:
    f.write(raw)
return "You uploaded %s (%d bytes)." % (filename, len(raw))

It returns the proper amount of bytes every single time.

The upload works fine for file like .txt, .php, .css ...

But it results in a corrupted file for other files like .jpg, .png, .pdf, .xls ...

I tried to change the open() function

with open(filename, 'wb') as f:

It returns the following error:

TypeError('must be bytes or buffer, not str',)

I guess its an issue related to binary files ?

Is there something to install on top of Python to run upload for any file type ?

---

Update

Just to be sure, as pointed out by @thkang I tried to code this using the dev version of bottlepy and the built-in method .save()

upload = request.files.get('upload')
upload.save(upload.filename)

It returns the exact same Exception error

TypeError('must be bytes or buffer, not str',)

---

Update 2

Here the final code which "works" (and dont pop the error TypeError('must be bytes or buffer, not str',)

upload = request.files.get('upload')
raw = upload.file.read().encode()
filename = upload.filename
with open(filename, 'wb') as f:
    f.write(raw)

Unfortunately, the result is the same : every .txt file works fine, but other files like .jpg, .pdf ... are corrupted

I've also noticed that those file (the corrupted one) have a larger size than the orginal (before upload)

This binary thing must be the issue with Python 3x

Note :

• I use python 3.1.3

• I use BottlePy 0.11.6 (raw bottle.py file, no 2to3 on it or anything)

Answer 1

Try this:

upload = request.files.get('upload')

with open(upload.file, "rb") as f1:
    raw = f1.read()
    filename = upload.filename
    with open(filename, 'wb') as f:
        f.write(raw)

    return "You uploaded %s (%d bytes)." % (filename, len(raw))

Update

Try value:

# Get a cgi.FieldStorage object
upload = request.files.get('upload')

# Get the data
raw = upload.value;

# Write to file
filename = upload.filename
with open(filename, 'wb') as f:
    f.write(raw)

return "You uploaded %s (%d bytes)." % (filename, len(raw))

Update 2

See this thread, it seems to do same as what you are trying...

# Test if the file was uploaded
if fileitem.filename:

   # strip leading path from file name to avoid directory traversal attacks
   fn = os.path.basename(fileitem.filename)
   open('files/' + fn, 'wb').write(fileitem.file.read())
   message = 'The file "' + fn + '" was uploaded successfully'

else:
   message = 'No file was uploaded'

Answer 2

In Python 3x all strings are now unicode, so you need to convert the read() function used in this file upload code.

The read() function returns a unicode string aswell, which you can convert into proper bytes via encode() function

Use the code contained in my first question, and replace the line

raw = upload.file.read()

with

raw = upload.file.read().encode('ISO-8859-1')

That's all ;)

Further reading : http://python3porting.com/problems.html