How to choose blob block size in Azure

Question

I want to use Append blobs in Azure storage. When Im uploading a blob, I should choose the block size. What I should consider when choosing the block size? I see no difference if Im uploading a file which has bigger size then block size. How to choose the right block size?

Answer 1

According to your description, I did some research, you could refer to it for a better understanding about blocks of append blob:

I just checked the CloudAppendBlob.AppendText and CloudAppendBlob.AppendFromFile. If the file size or text content size less than 4MB, then it would be uploaded to a new individual block. Here I used CloudAppendBlob.AppendText for appending text content (byte size less than 4MB) three times, you could refer to the network traces as follows:

For content size > 4MB, then the client SDK would divide the content into small pieces (4MB) and upload them into each blocks. Here I uploaded a file with the size about 48.8MB, you could refer to the network traces as follows:

As Gaurav Mantri mentioned that you could choose small block size for low speed network. Moreover, for small block size write, you would retrieve the better performance for write requests, but when you reading data, your data spans across multiple separate blocks, it would slow down your read requests. It depends on the write/read ratio your application expected, for optimal reads, I recommend that you need to batch writes to be as near 4MB as possible, which would bring you with slower write requests but reads to be much faster.

Answer 2

A few things to consider when deciding on the block size:

• In case of an Append Blob, maximum size of a block can be 4 MB so you can't go beyond that number.
• Again, a maximum of 50000 blocks can be uploaded so you would need to divide the blob size with 50000 to decide the size of a block. For example, if you're uploading a 100MB file and decide to choose 100 byte block, you would end up with 1048576 (100x1024x1024/100) blocks which is more than allowed limit of 50000 so it is not allowed.
• Most importantly, I believe it depends on your Internet speed. If you have a really good Internet connection, you can go up to 4MB block size. For not so good Internet connections, you can reduce the limit. For example, I always try to use 256-512KB block size as the Internet connection I have is not good.