Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Download all blobs files locally from azure container using python

I'm using python 3.6 in visual studio and I want to download all blobs files from my azure container in single folder. This is my code but the problem is, that it downloads 1 blob file in the folder and then when downloading the second file it overwrite the first file and in the end I only have the last blob in my local folder. How can I download all the blobs files at once in a single folder?

from azure.storage.blob import BlockBlobService
block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)

generator = block_blob_service.list_blobs(CONTAINER_NAME)
        for blob in generator:
            block_blob_service.get_blob_to_path(CONTAINER_NAME, blob.name, LOCAL_FILE)
like image 349
s.kaiser Avatar asked Mar 18 '17 14:03

s.kaiser


People also ask

How do I download an entire Azure container?

If you want, you can use the AzCopy tool to download an entire blob container. Assuming you have the latest version of the Azure SDK installed, you can find this tool in the C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy folder.


1 Answers

Based on my understanding, I think there are two solutions for your needs.

  1. Download all blobs from a container, and write these blob content to a single file via the method get_blob_to_bytes or get_blob_to_stream, please see my sample code as below.

    from azure.storage.blob import BlockBlobService
    
    block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)
    
    generator = block_blob_service.list_blobs(CONTAINER_NAME)
    
    fp = open('<your-local-file-name>', 'ab')
    
    for blob in generator:
        # Using `get_blob_to_bytes`
        b = service.get_blob_to_bytes(container_name, blob.name)
        fp.write(b.content)
        # Or using `get_blob_to_stream`
        # service.get_blob_to_stream(container_name, blob.name, fp)
    
    fp.flush()
    fp.close()
    
  2. Download all blobs from a container, and write these blobs into a zip file via the method get_blob_to_bytes, please see my sample code below.

    from azure.storage.blob import BlockBlobService
    import zipfile
    
    block_blob_service = BlockBlobService(account_name=ACCOUNT_NAME, account_key=ACCOUNT_KEY)
    
    generator = block_blob_service.list_blobs(CONTAINER_NAME)
    
    zf = zipfile.ZipFile(CONTAINER_NAME+'.zip', 
                 mode='w',
                 compression=zipfile.ZIP_DEFLATED, 
                 )
    
    for blob in generator:
        b = service.get_blob_to_bytes(container_name, blob.name)
        zf.writestr(blob.name, b.content)
    
    zf.close()
    

Hope it helps. Any concern, please feel free to let me know.

like image 102
Peter Pan Avatar answered Sep 20 '22 05:09

Peter Pan