What would be the best way to search an IReliableDictionary?

Question

Based on these posts:

Convert IReliableDictionary to IList

What is the most optimal method of querying a reliable dictionary collection

I should be able to use Linq to query an IReliableDictionary, but it would appear that this interface no longer implements IEnumerable and the Linq extensions are not available. At least in version 5.0.0.0 of the Microsoft.ServiceFabric.Data.Interfaces assembly.

If this is true, what would be the best way to search an IReliableDictionary?

Answer 1

Yes, we did remove IEnumerable from Reliable Collections in the GA release. As Allan T mentions, Reliable Collections aren't really the same as regular .NET collections, even though they represent the same fundamental data structures. One of the big differences you've probably noticed is that all operations are asynchronous, because of locking semantics and I/O for replication and disk reads. It's the latter part that drove the decisions to remove IEnumerable, because it is strictly synchronous and we're not. Instead we now use IAsyncEnumerable, which as of yet does not support the full set of LINQ extension methods.

We're working on asynchronous LINQ extension methods, but in the meantime, there are a couple of ways to work with IAsyncEnumerable.

Eli Arbel has async extension methods on Gist that provide a bridge to System.Interactive.Async and also async implementations of Select, SelectMany, and Where.

Or you can wrap IAsyncEnumerable in a regular IEnumerable that simply wraps the asynchronous calls with synchronous methods, and that will give you the full set of LINQ extensions methods again. Then you can use the extension method in a regular LINQ query:

        using (ITransaction tx = this.StateManager.CreateTransaction())
        {
            var x = from item in (await clusterDictionary.CreateEnumerableAsync(tx)).ToEnumerable()
                   where item.Value.Status == ClusterStatus.Ready
                   orderby item.Value.CreatedOn descending
                   select new ClusterView(
                       item.Key,
                       item.Value.AppCount,
                       item.Value.ServiceCount,
                       item.Value.Users.Count(),
                       this.config.MaximumUsersPerCluster,
                       this.config.MaximumClusterUptime - (DateTimeOffset.UtcNow - item.Value.CreatedOn.ToUniversalTime()));

        }

Answer 2

I don't know if it's the "best" way, but I have been using the following

public static async Task<IList<KeyValuePair<Guid, T>>> QueryReliableDictionary<T>(IReliableStateManager stateManager, string collectionName, Func<T, bool> filter)
{
    var result = new List<KeyValuePair<Guid, T>>();

    IReliableDictionary<Guid, T> reliableDictionary =
        await stateManager.GetOrAddAsync<IReliableDictionary<Guid, T>>(collectionName);

    using (ITransaction tx = stateManager.CreateTransaction())
    {
        IAsyncEnumerable<KeyValuePair<Guid, T>> asyncEnumerable = await reliableDictionary.CreateEnumerableAsync(tx);
        using (IAsyncEnumerator<KeyValuePair<Guid, T>> asyncEnumerator = asyncEnumerable.GetAsyncEnumerator())
        {
            while (await asyncEnumerator.MoveNextAsync(CancellationToken.None))
            {
                if (filter(asyncEnumerator.Current.Value))
                    result.Add(asyncEnumerator.Current);
            }
        }
    }
    return result;
}

You use the method by passing in the StateManager, the name of the collection you wish to query and a lambda function with your query logic. For example:

var queryResult = await QueryReliableDictionary<string>(this.StateManager, "CustomerCollection", name => !string.IsNullOrWhiteSpace(name) && (name.IndexOf("fred", StringComparison.OrdinalIgnoreCase) >= 0));