Azure blob storage limitation and filter

Question

Azure storage for multi-tenant application. We are working on to develop a multi-tenant application on Azure, with approximate 10,000 tenants and approximate 100 GB to 1 TB data storage is required per tenant. The application is to maintain the documents and binary content along with the metadata for each tenant separately. We are thinking towards Azure Block Blob storage to store the data. Since, the requirement is to maintain the data separate for each tenant, we came across with the following approach.

• Create a separate storage account for each tenant
• That helps to maintain the usage tenant wise, which again helps on billing as well
• Create a separate container in each storage account to segregate based on category
• Store document in block blob storage along with the metadata.

We have following queries with respect to the our approach:

1. Is it good idea to store documents or binary content in block blob along with the metadata? Or is there any better way of achieving it (probably using SQL Azure for metadata and blob for content, or better)?
2. How to query the data with some filter condition on metadata? i.e. retrieve all blob where metadat1 = value1 and metadata2=value2
3. Is it good idea to create a separate storage account for each tenant?
   a. If not, then what would be the model thru which we can store tenant specific data in the Azure storage and application can efficiently use them?
4. Is there bandwidth or any other limitation on number of request to read/write data on Blob storage in context of scalability and high availability?
5. As per the azure pricing model, they charge slab wise for the storage, i.e. first 1 TB $0.095 / GB, next 49 TB $0.08 / GB. This charges are application on per storage account or on per subscription?
   a. Same way, transaction cost is applicable on per storage account or on per subscription?

Answer 1

1. Is it good idea to store documents or binary content in block blob along with the metadata? Or is there any better way of achieving it (probably using SQL Azure for metadata and blob for content, or better)?
2. How to query the data with some filter condition on metadata? i.e. retrieve all blob where metadat1 = value1 and metadata2=value2

To answer 1 and 2, you can't query on metadata in blob storage. So I guess your best option would be to use SQL Azure or Azure Table Storage as both of them have querying capabilities. Given that you'll be storing huge number of blobs (and thus even more metadata), I'm more inclined towards table storage but that would require special design considerations like proper partitioning.

Is it good idea to create a separate storage account for each tenant? a. If not, then what would be the model thru which we can store tenant specific data in the Azure storage and application can efficiently use them?

I can think of 3 reasons why having a separate storage account per tenant is a good idea:

1. It simplifies your billing.
2. It will help you maintain scalability targets.
3. Since you mentioned that each tenant can potentially store up to 1 TB of data, given the current storage account limit of 200 TB, you can only maintain a maximum of 200 tenants per storage account. After that you would need to find another storage account and start storing the data there.

All in all a much elegant solution keeping separate storage account / tenant. The challenge would be to have the default limit increased from 20 storage accounts / subscription. You would need to chat with support for that.

Is there bandwidth or any other limitation on number of request to read/write data on Blob storage in context of scalability and high availability?

Yes, Please read the scalability targets blog from Windows Azure Blob Storage team: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/10/windows-azure-storage-abstractions-and-their-scalability-targets.aspx

As per the azure pricing model, they charge slab wise for the storage, i.e. first 1 TB $0.095 / GB, next 49 TB $0.08 / GB. This charges are application on per storage account or on per subscription? a. Same way, transaction cost is applicable on per storage account or on per subscription?

Not sure about this but I am guessing it's per storage account. You may want to contact support for this.

Hope this helps.