How to add thousands of items to a binded collection without locking GUI

Question

I have a setup where potentially thousands of items (think 3000-5000) will be added to an ObservableCollection that is binded to some visual interface. Currently, the process of adding them is quite slow (approx. 4 seconds/1000 items), and of course the GUI is unresponsive during that time. What is a good method to handle moving that many items at once into a collection without worrying about the system locking up? I've looked at DispatcherTimer but I'm not sure if it will provide everything I need it to.

Another question - Is there something I can do to speed up the creation of these objects so that it doesn't take so long to add them to the collection? Currently I use them like so: Collection.Add(new Item(<params>)) Would generating the items beforehand, in a background thread probably, decrease the time it takes to add them by a noticeable amount?

Edit: Virtualization is not possible. The requirements specify a WrapPanel look, so the display is actually a ListBox which has a templated ItemsPanel

Edit2: According to the stopwatch, the bottleneck is actually putting items into my ObservableCollection. I will try changing that collection type and doing my own notification to see if that speeds it up substantially.

Edit3: So the answer is in one place - I solved this issue (with help from below) by creating a class which inherits from ObservableCollection. This class did two things - expose a method to add collections at one time, and added the ability to suppress the CollectionChanged Event. With these changes the time it takes to add 3000 items is roughly .4 seconds (97% improvement). This link details the implementation of these changes.

Answer 1

You've said 1000, so I'll stick to that number just for instance.

IIRC, the observable collection has a small drawback - if you add the items one by one, it raises notifies once per each item. That means that you have 1000 notifications for 1000 of items and the UI thread will run at deadly speed just to keep up with redrawing the screen.

Do you need to redraw ASAP? Maybe you can batch the additions? Split the 1000 of items into a few packed of 100 items, or a little more packets of 50 or 20 items. Then, instead of putting all items one by one, put them in packets. But beware: you have to use some methods like AddRange implemented by the collection it self, not by LINQ, or else you will again have one-by-one insertion. If you find such method, it should cut the number of events significantly, because the collection should raise the Changed event only once per AddRange call.

If observable collection does not have AddRange, either use different collection, or write your own, just a wrapper will probably be sufficient. The goal is to NOT raise Changed event at every single Add(), but after a reasonable count of them, or - maybe just skip raising Changed when items are added and raise Changed at some regular time intervals? This would be beneficial especially, if your data "flows in" indefinitely at a constant rate.

Of course, at that number of items coming onto the screen, you may just as well be held at the rendering it self. If your ItemTemplates are complicated, a 1000 of objects times 1000 of instances of visual layers/properties may simply kill the user experience. Have you simplified the ItemTemplates to the bare minimum?

Last thing: consider using virtualizing StackPanels as the ItemPanels in your ItemsControl/ListBoxes. It can greatly reduce the memory footprint and the number of items drawn at a single point of time. This will not necessarily help in the number or events raised, but it may help greatly when you have complex item templates!

Edit: you are using ObservableCollection, so I've assumed WPF/Silverlight.. update the question if this is not correct

Answer 2

WPF Binding supports concurrency for this reason. Try setting Binding.IsAsync to true. In addition.

• Don't use ObservableCollection<T>, it is slow for this because each time an item is added it raises events. Use something faster like List<T> and raise your property change notification after all your items are added.
• Pre-create your items in a background thread, then push them into your collection.
• Check other parts of code involved to see if there is bloat, and trim.

Answer 3

By request, here is how I solved this issue. I started by creating a class which inherits from ObservableCollection. This class did two things - expose a method to add entire collections at once, and added the ability to suppress the CollectionChanged Event. With these changes the time it takes to add 3000 items is roughly .4 seconds (97% improvement). This link details the implementation of these changes.