Custom ObservableCollection<T> or BindingList<T> with support for periodic notifications

Question

Summary

I have a large an rapidly changing dataset which I wish to bind to a UI (Datagrid with grouping). The changes are on two levels;

• Items are frequently added or removed from the collection (500 a second each way)
• Each item has a 4 properties which will change up to 5 times in its lifetime

The characteristics of the data are as follows;

• There are ~5000 items in the collection
• An item may, within a second, be added then have 5 property changes and then be removed.
• An item may also remain in some interim state for a while and should be displayed to the user.

The key requirement which I'm having problems with;

• The user should be able to sort the dataset by any property on the object

What I would like to do;

• Update the UI only every N seconds
• Raise only the relevant NotifyPropertyChangedEvents

If item 1 has a property State which moves from A -> B -> C -> D in the interval I need/want only one 'State' change event to be raised, A->D.

I appreciate a user doesn't need to have the UI updated thousands of times a second. if an item is added, has its state changed and is removed all within the window of N seconds between UI updates it should never hit the DataGrid.

DataGrid

The DataGrid is the component which I am using to display the data. I am currently using the XCeed DataGrid as it provides dynamic grouping trivially. I am not emotionally invested in it, the stock DataGrid would be fine if I could provide some dynamic grouping options (Which includes the properties which change frequently).

The bottleneck in my system is currently in the time taken to re-sort when an item's properties change

This takes 98% of CPU in the YourKit Profiler.

A different way to phrase the question

Given two BindingList / ObservableCollection instances which were initially identical but the first list has since had a series of additional updates (which you can listen for), generate the minimal set of changes to turn one list into the other.

External Reading

What I need is an equivalent of this ArrayMonitor by George Tryfonas but generalized to support adding and removing of items (they will never be moved).

NB I would really appreciate someone editing the title of the question if they can think of a better summary.

EDIT - My Solution

The XCeed grid binds the cells directly to the items in the grid whereas the sorting & grouping functionality is driven by the ListChangedEvents raised on the BindingList. This is slightly counter intuitive and ruled out the MontioredBindingList below as the rows would update before the groups.

Instead I wrap the items themselves, catching the Property changed events and storing them in a HashSet as Daniel suggested. This works well for me, I periodically iterate over the items and ask them to notify of any changes.

MonitoredBindingList.cs

Here is my attempt at a binding list which can be polled for update notifications. There are likely some bugs with it as it was not useful to me in the end.

It creates a queue of Add/Remove events and keeps track of changes via a list. The ChangeList has the same order as the underlying list so that after we've notified of the add/remove operations you can raise the changes against the right index.

/// <summary>
///  A binding list which allows change events to be polled rather than pushed.
/// </summary>
[Serializable]

public class MonitoredBindingList<T> : BindingList<T>
{
    private readonly object publishingLock = new object();

    private readonly Queue<ListChangedEventArgs> addRemoveQueue;
    private readonly LinkedList<HashSet<PropertyDescriptor>> changeList;
    private readonly Dictionary<int, LinkedListNode<HashSet<PropertyDescriptor>>> changeListDict;

    public MonitoredBindingList()
    {
        this.addRemoveQueue = new Queue<ListChangedEventArgs>();
        this.changeList = new LinkedList<HashSet<PropertyDescriptor>>();
        this.changeListDict = new Dictionary<int, LinkedListNode<HashSet<PropertyDescriptor>>>();
    }

    protected override void OnListChanged(ListChangedEventArgs e)
    {
        lock (publishingLock)
        {
            switch (e.ListChangedType)
            {
                case ListChangedType.ItemAdded:
                    if (e.NewIndex != Count - 1)
                        throw new ApplicationException("Items may only be added to the end of the list");

                    // Queue this event for notification
                    addRemoveQueue.Enqueue(e);

                    // Add an empty change node for the new entry
                    changeListDict[e.NewIndex] = changeList.AddLast(new HashSet<PropertyDescriptor>());
                    break;

                case ListChangedType.ItemDeleted:
                    addRemoveQueue.Enqueue(e);

                    // Remove all changes for this item
                    changeList.Remove(changeListDict[e.NewIndex]);
                    for (int i = e.NewIndex; i < Count; i++)
                    {
                        changeListDict[i] = changeListDict[i + 1];
                    }

                    if (Count > 0)
                        changeListDict.Remove(Count);
                    break;

                case ListChangedType.ItemChanged:
                    changeListDict[e.NewIndex].Value.Add(e.PropertyDescriptor);
                    break;
                default:
                    base.OnListChanged(e);
                    break;
            }
        }
    }

    public void PublishChanges()
    {
        lock (publishingLock)
            Publish();
    }

    internal void Publish()
    {
        while(addRemoveQueue.Count != 0)
        {
            base.OnListChanged(addRemoveQueue.Dequeue());
        }

        // The order of the entries in the changeList matches that of the items in 'this'
        int i = 0;
        foreach (var changesForItem in changeList)
        {
            foreach (var pd in changesForItem)
            {
                var lc = new ListChangedEventArgs(ListChangedType.ItemChanged, i, pd);
                base.OnListChanged(lc);
            }
            i++;
        }
    }
}

Answer 1

We are talking about two things here:

1. The changes to the collection. This raises the event INotifyCollectionChanged.CollectionChanged
2. The changes to the properties of the items. This raises the event INotifyPropertyChanged.PropertyChanged

The interface INotifyCollectionChanged needs to be implemented by your custom collection. The interface INotifyPropertyChanged needs to be implemented by your items. Furthermore, the PropertyChanged event only tells you which property was changed on an item but not what was the previous value.
This means, your items need to have a implementation that goes something like this:

• Have a timer that runs every N seconds
• Create a HashSet<string> that contains the names of all properties that have been changed. Because it is a set, each property can only be contained one or zero times.
• When a property is changed, add its name to the hash set if it is not already in it.
• When the timer elapses, raise the PropertyChanged event for all properties in the hash set and clear it afterwards.

Your collection would have a similar implementation. It is however a little bit harder, because you need to account for items that have been added and deleted between to timer events. This means, when an item is added, you would add it to a hash set "addedItems". If an item is removed, you add it to a "removedItems" hash set, if it is not already in "addedItems". If it is already in "addedItems", remove it from there. I think you get the picture.

To adhere to the principle of separation of concerns and single responsibility, it would be even better to have your items implement INotifyPropertyChanged in the default way and create a wrapper that does the consolidation of the events. That has the advantage that your items are not cluttered with code that doesn't belong there and this wrapper can be made generic and used for every class that implements INotifyPropertyChanged.
The same goes for the collection: You can create a generic wrapper for all collections that implement INotifyCollectionChanged and let the wrapper do the consolidation of the events.