Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Azure Storage Calculated MD5 does not match existing property

I'm trying to pass an Azure Storage blob through an ashx. On the blockBlob.DownloadToStream(memoryStream) it's throwing the following Exception: Microsoft.WindowsAzure.Storage.StorageException: Calculated MD5 does not match existing property

I know it's finding the correct blob. If I put in a container and path that don't exist then it gives me a 404 exception instead.

I've Googled for hints on what might be causing this error but nothing useful is coming up. Does anyone have any thoughts on what might be causing this? I've rewritten this code a couple different ways over the last couple days but it always dies on DownloadToStream.

using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Blob;

public void ProcessRequest(HttpContext context) {
    // Retrieve storage account from connection string.
    Microsoft.WindowsAzure.Storage.CloudStorageAccount storageAccount = Microsoft.WindowsAzure.Storage.CloudStorageAccount.Parse(Microsoft.WindowsAzure.CloudConfigurationManager.GetSetting("StorageConnectionString"));

    // Create the blob client.
    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

    // Retrieve reference to a previously created container.
    CloudBlobContainer container = blobClient.GetContainerReference("gmt");

    // Retrieve reference to blob named "articles/142/222.jpg".
    CloudBlockBlob blockBlob = container.GetBlockBlobReference("articles/142/222.jpg");

    using (var memoryStream = new MemoryStream()) {
        blockBlob.DownloadToStream(memoryStream);
        byte[] photoByte = ReadFully(memoryStream);
        context.Response.Clear();
        context.Response.ContentType = "image/jpeg";
        context.Response.OutputStream.Write(photoByte, 0, photoByte.Length);
    }
}

public static byte[] ReadFully(Stream input) {
    input.Position = 0;
    using (MemoryStream ms = new MemoryStream()) {
        input.CopyTo(ms);
        return ms.ToArray();
    }
}
like image 731
Ryan Vettese Avatar asked Dec 12 '13 14:12

Ryan Vettese


People also ask

What is content MD5 in Azure?

Azure stores the file hash value in the CONTENT-MD5 tag as a base64 encoded representation of the binary MD5 hash value, whereas when we compute the MD5 value locally, the output of the hash is a hex representation of the binary hash value…

Is Azure Blob Storage deprecated?

Azure Blob storage file source with Azure Queue Storage (Deprecated) - Azure Databricks | Microsoft Learn. This browser is no longer supported. Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.

How do I sync blob storage?

You can synchronize local storage with Azure Blob storage by using the AzCopy v10 command-line utility. You can synchronize the contents of a local file system with a blob container. You can also synchronize containers and virtual directories with one another. Synchronization is one way.


2 Answers

I had this problem on my local DEV environment. And it seems that db of AzureStorageEmulator got corrupted.

The solution (for local env!):

  • drop the emulator's db (e.g. AzureStorageEmulatorDb57)
  • run AzureStorageEmulator.exe init -sqlinstance . (you may need to customize the instance name)
  • run AzureStorageEmulator.exe start
  • restart the application, so it gets a new handler to the emulator
like image 93
andrew.fox Avatar answered Nov 15 '22 20:11

andrew.fox


I was able to recreate the problem you're facing. This happens if the Content MD5 property of the blob is somehow corrupted. I had a blob with some content MD5 (which was correct). I then programmatically changed the MD5 to some other value (which is incorrect). Now when I call DownloadToStream() method on the blob, I get exact same error.

You can bypass this check by setting DisableContentMD5Validation to true in BlobRequestOptions as shown in the code below:

            BlobRequestOptions options = new BlobRequestOptions()
            {
                DisableContentMD5Validation = true,
            };
            blockBlob.DownloadToStream(memoryStream, null, options);

Give it a try and it should work.

On a side note, you may want to modify your ReadFully method as well. You would need to move the input stream pointer to the beginning.

    public static byte[] ReadFully(Stream input)
    {
        input.Position = 0;//Positioning it to the top of stream.
        using (MemoryStream ms = new MemoryStream())
        {
            input.CopyTo(ms);
            return ms.ToArray();
        }
    }
like image 24
Gaurav Mantri Avatar answered Nov 15 '22 21:11

Gaurav Mantri