Python Azure blob storage upload file bigger then 64 MB

Question

From the sample code, I can upload 64MB, without any problem:

   myblob = open(r'task1.txt', 'r').read()
   blob_service.put_blob('mycontainer', 'myblob', myblob, x_ms_blob_type='BlockBlob')

What if I want to upload bigger size?

Thank you

Answer 1

I ran into the same problem a few days ago, and was lucky enough to find this. It breaks up the file into chunks and uploads it for you.

I hope this helps. Cheers!

Answer 2

I'm not a Python programmer. But a few extra tips I can offer (my stuff is all in C):

Use HTTP PUT operations(comp=block option) for as many Blocks (4MB each) as required for your file, and then use a final PUT Block List (comp=blocklist option) that coalesces the Blocks. If your Block uploads fail or you need to abort, the cleanup for deleting the partial set of Blocks previously uploaded is a DELETE command for the file you are looking to create, but this appears supported by the 2013-08-15 version only (Someone from the Azure support should confirm this).

If you need to add Meta information, an additional PUT operation (with the comp=metadata prefix) is what I do when using the Block List method. There might be a more efficient way to tag the meta information without requiring an additional PUT, but I'm not aware of it.