Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Writing to Azure Block Blobs

Tags:

azure

I am using PutBlock and PutBlockList to upload data to a block blob, the code i am using for this is below:-

CloudBlobContainer container = blobStorage.GetContainerReference("devicebackups");
var permissions = container.GetPermissions();
permissions.PublicAccess = BlobContainerPublicAccessType.Container;
container.SetPermissions(permissions);
CloudBlockBlob blob = container.GetBlockBlobReference(serialNo.ToLower() + " " + dicMonths[DateTime.Now.Month]);
try
{
    var serializer = new XmlSerializer(typeof(List<EnergyData>));
    var stringBuilder = new StringBuilder();
    using (XmlWriter writer = XmlWriter.Create(stringBuilder))
    {
        try
        {
            serializer.Serialize(writer, deviceData);
            byte[] byteArray = Encoding.UTF8.GetBytes(stringBuilder.ToString());

            List<string> blockIds = new List<string>();
            try 
            { 
                blockIds.AddRange(blob.DownloadBlockList(BlockListingFilter.Committed).Select(b => b.Name)); 
            }
            catch (StorageClientException e)
            {
                if (e.ErrorCode != StorageErrorCode.BlobNotFound)
                {
                    throw;
                }
                blob.Container.CreateIfNotExist();
            }
            var newId = Convert.ToBase64String(Encoding.UTF8.GetBytes(blockIds.Count().ToString()));
            blob.PutBlock(newId, new MemoryStream(byteArray), null);
            blockIds.Add(newId);
            blob.PutBlockList(blockIds);
        }
        catch (Exception ex) 
        { 
            UT.ExceptionReporting(ex, "Error in Updating Backup Blob - writing byte array to blob"); 
        }
    }
}
catch (Exception ex) 
{ 
    UT.ExceptionReporting(ex, "Error in Updating Backup Blob - creating XmlWriter"); 
}
}
catch (Exception ex) 
{ 
    UT.ExceptionReporting(ex, "Error in Updating Backup Blob - getting container and blob references, serial no -" + serialNo); 
}

This works for 10 blocks, then on the 11th block it crashes with the following error:-

StorageClientException - The specified block list is invalid.

InnerException = {"The remote server returned an error: (400) Bad Request."}

I have searched the internet for reports of the same error, but had no luck.

Any help would be much appreciated.

like image 760
ChrisW Avatar asked Apr 23 '12 12:04

ChrisW


People also ask

How do you block a blob?

A block blob can include up to 50,000 blocks. Each block in a block blob can be a different size, up to the maximum size permitted for the service version in use. To create or modify a block blob, write a set of blocks via the Put Block operation and then commit the blocks to a blob with the Put Block List operation.

Can you text blob storage?

Azure Storage supports three types of blobs: Block blobs store text and binary data. Block blobs are made up of blocks of data that can be managed individually. Block blobs can store up to about 190.7 TiB.

What is block in Azure blob?

Block blobs are composed of blocks and are ideal for storing text or binary files, and for uploading large files efficiently. Append blobs are also made up of blocks, but they are optimized for append operations, making them ideal for logging scenarios.


2 Answers

For a given blob, the length of the value specified for the blockid parameter must be the same size for each block.

http://msdn.microsoft.com/en-us/library/windowsazure/dd135726.aspx

The first 10 blocks are numbered 0 through 9. The 11th block is number 10, which is longer by one character. So you should change your numbering scheme to always use the same length. One solution would be to convert the count to a zero-padded string that's long enough to hold the number of blocks you expect to have.

But if you don't need the benefits of using blocks, you're probably better off just writing the whole blob in one go instead of using blocks.

like image 155
Brian Reischl Avatar answered Sep 30 '22 04:09

Brian Reischl


Set your BlockID has below code

var blockIdBase64 = Convert.ToBase64String(Encoding.UTF8.GetBytes(blockId.ToString(CultureInfo.InvariantCulture).PadLeft(32, '0')));
like image 25
user145610 Avatar answered Sep 30 '22 04:09

user145610