Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Copy all Rows to another Table in Azure Table Storage

What is the best way to copy all the rows from one table to another table?
I tried below code to get all the rows in a table:

    TableServiceContext _dataContext;
    public IEnumerable<T> GetAllEntities()
    {
        IQueryable<T> query = null;
        try
        {
            query = _dataContext.CreateQuery<T>(_tableName);
        }
        catch (Exception ex)
        {

        }
        return query.ToArray();
    }

but it doesnt get rows more than around 900. I have few hundreds of thousands of rows.
Updated Code:

 public class TableRepository<T> : IRepository<T> 
    where T : TableEntity
{
    protected readonly string _tableName;
    protected readonly TableServiceContext _dataContext;
    protected readonly CloudTable _tableReference;

    public TableRepository(string tableName, CloudTableClient tableClient)
    {
        _tableName = tableName;
        _dataContext = tableClient.GetTableServiceContext();
        _tableReference = tableClient.GetTableReference(tableName);
        _dataContext.ResolveType = ResolveEntityType;
        _dataContext.MergeOption = System.Data.Services.Client.MergeOption.NoTracking;
    }

    public IEnumerable<T> GetAllEntities()
    {
        List<T> allEntities = new List<T>();
        try
        {
            Microsoft.WindowsAzure.Storage.Table.TableContinuationToken tableContinuationToken = null;
            do
            {
                var queryResponse = _tableReference.ExecuteQuerySegmented<T>(null, tableContinuationToken, null, null);
                tableContinuationToken = queryResponse.ContinuationToken;
                allEntities.AddRange(queryResponse.Results);
            }
            while (tableContinuationToken != null);

        }
        catch (Exception ex)
        {
            throw new DALException(_tableName,_dataContext.BaseUri.OriginalString, "An error occured while querying data", ex);
        }
        return allEntities;
    }

}

but with error:

Error 121 'T' must be a non-abstract type with a public parameterless constructor in order to use it as parameter 'TElement' in the generic type or method 'Microsoft.WindowsAzure.Storage.Table.CloudTable.ExecuteQuerySegmented

like image 982
Srinivas Avatar asked Aug 22 '13 09:08

Srinivas


People also ask

How do I export data from Azure table storage?

You can use Azure Storage Explorer. It is free and supported by Microsoft. Browse to the appropriate storage account, click on the table storage you want to export and look for the export option in the explorer.

How do I import data into Azure Table storage?

Use an account key The following properties are supported. The type property must be set to AzureTableStorage. Specify the information needed to connect to Storage for the connectionString property. You can also put account key in Azure Key Vault and pull the accountKey configuration out of the connection string.


1 Answers

The reason you're getting only 900 results back is because you're running into continuation tokens. By default a single request to table service will return a maximum of 1000 entities. It could be less than 1000 entities (and even 0) but never more than 1000. If there are more entities available, then the table service returns a continuation token which should be used to fetch next set of entities.

So your code should look for the continuation token and should keep on fetching the entities till the time token is returned by table service. Do take a look at the sample code below:

private IEnumerable<T> FetchAllEntities()
{
    List<T> allEntities = new List<T>();
    CloudStorageAccount storageAccount = CloudStorageAccount.DevelopmentStorageAccount;
    CloudTable table = storageAccount.CreateCloudTableClient().GetTableReference("MyTable");
    Microsoft.WindowsAzure.Storage.Table.TableContinuationToken tableContinuationToken = null;
    do
    {
        var queryResponse = table.ExecuteQuerySegmented<T>(null, tableContinuationToken, null, null);
        tableContinuationToken = queryResponse.ContinuationToken;
        allEntities.AddRange(queryResponse.Results);
    }
    while (tableContinuationToken != null);
    return allEntities;
}

Update

For your error, try changing the following

public class TableRepository<T> : IRepository<T> 
    where T : TableEntity

to

public class TableRepository<T> : IRepository<T> 
    where T : TableEntity, new()

notice the addition of new() after TableEntity.

like image 200
Gaurav Mantri Avatar answered Sep 30 '22 12:09

Gaurav Mantri