Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Batch delete in Windows Azure Table Storage

I'm trying to delete all entities between 2 dates (i.e. 2 partition keys) in the WADLogsTable. So far, the best code I've found is this one (from https://www.wintellect.com/deleting-entities-in-windows-azure-table-storage):

private static void DeleteAllEntitiesInBatches(CloudTable table, Expression<Func<DynamicTableEntity, bool>> filters)
        {
            Action<IEnumerable<DynamicTableEntity>> processor = entities =>
            {
                var batches = new Dictionary<string, TableBatchOperation>();

                foreach (var entity in entities)
                {
                    TableBatchOperation batch = null;

                    if (batches.TryGetValue(entity.PartitionKey, out batch) == false)
                    {
                        batches[entity.PartitionKey] = batch = new TableBatchOperation();
                    }

                    batch.Add(TableOperation.Delete(entity));

                    if (batch.Count == 100)
                    {
                        table.ExecuteBatch(batch);
                        batches[entity.PartitionKey] = new TableBatchOperation();
                    }
                }

                foreach (var batch in batches.Values)
                {
                    if (batch.Count > 0)
                    {
                        table.ExecuteBatch(batch);
                    }
                }
            };

            ProcessEntities(table, processor, filters);
        }

        private static void ProcessEntities(CloudTable table, Action<IEnumerable<DynamicTableEntity>> processor, Expression<Func<DynamicTableEntity, bool>> filters)
        {
            TableQuerySegment<DynamicTableEntity> segment = null;

            while (segment == null || segment.ContinuationToken != null)
            {
                if (filters == null)
                {
                    segment = table.ExecuteQuerySegmented(new TableQuery().Take(100), segment == null ? null : segment.ContinuationToken);
                }
                else
                {
                    var query = table.CreateQuery<DynamicTableEntity>().Where(filters).Take(100).AsTableQuery();
                    segment = query.ExecuteSegmented(segment == null ? null : segment.ContinuationToken);
                }

                processor(segment.Results);
            }
        }

But I don't know what I should pass as the "filters" parameter. I came up with this rather naive attempt:

Expression<Func<DynamicTableEntity, bool>> filters = e => long.Parse(e.PartitionKey) >= startTicks && long.Parse(e.PartitionKey) <= endTicks;

But that doesn't work. At runtime, I get the following error:

The expression ((Parse([10007].PartitionKey) >= 635109048000000000) And (Parse([10007].PartitionKey) <= 635115960000000000)) is not supported.

I don't know if my problem is related to Azure tables or Expression but any help will be appreciated.

EDIT: I'd be happy to provide with more info if needed.

like image 716
Rodolphe Avatar asked Mar 27 '14 10:03

Rodolphe


People also ask

How do you clear a table storage in Azure?

If it is important for you to keep using the table, the only other option is to delete entities. For faster deletes, you can look at deleting entities using Entity Batch Transactions . But for deleting entities, you would need to first fetch the entities.

What is table storage in Windows Azure?

Azure Table storage is a service that stores non-relational structured data (also known as structured NoSQL data) in the cloud, providing a key/attribute store with a schemaless design. Because Table storage is schemaless, it's easy to adapt your data as the needs of your application evolve.

What is the maximum storage capacity of an azure table?

There is no limit on file shares you can create within a premium account. In a Premium FileStorage account, storage size is limited to 100 TB. You can perform up to 100,000 I/O operations per second.


1 Answers

I got an answer on another forum:

var startTicks = string.Format("{0:0000000000000000000}", start);
var endTicks = string.Format("{0:0000000000000000000}", end);
Expression<Func<DynamicTableEntity, bool>> filters = e => e.PartitionKey.CompareTo(startTicks) >= 0 && e.PartitionKey.CompareTo(endTicks) <= 0;
like image 176
Rodolphe Avatar answered Sep 29 '22 05:09

Rodolphe