Should I compress in-memory C# objects for better performance?

Question

I have an application (C#, WPF) that displays many financial charts with live data streaming from server. The data that is collected in-memory may grow to be a bit large, and I don't want to keep any data on disk.

Since the historical data itself doesn't change, but only added to, will it make sense to keep that data (which is stored in a collection object) in some compressed format?

Is it possible and can anyone recommend a good practice for it if so?

UPDATE

Some notes about performance and tradeoff: I am aware that compression will add a delay accessing the data, but, the user only needs fast updates on new data arriving. When accessing the data that was already rendered (for example, to study or re-render it) he doesn't require a quick response.

Answer 1

Compressing and decompressing will make your application slower so for performance (speed) it is not a good option. Compression is only useful when you are worried about available memory. It might be easier to store/swap the data to a temp folder.

The key to performance is measuring. Only take action when you have crunched the numbers.

Answer 2

Compressing the data has advantages in terms of memory usage, but disadvantages in terms of making the data unusable (you'll have to decompress it to use it again), as well as taking up extra CPU.

The tradeoff point where this would become beneficial is difficult to know without a lot more information - it's up to you. However, if you're not using this old, stale data, it may be better to just throw it away (ie: let it go out of scope/stop storing it) instead of compressing it.

Compression can be done via the classes in System.IO.Compression, and is fairly easy. These classes, in general, don't perform very well, however, so you may want to also consider a third party alternative, such as DotNetZip.

Answer 3

It's a trade off between performance and memory footprint and also depends on the data structures you are using. "Generic" compression (ie. gzip, run-length encoding etc.) doesn't make sense for many types of data.

One approach that might be applicable to you is picking a more appropriate data structure that optimizes memory footprint - i.e. for your chart do you really have to store independent stock prices or can you live by just storing delta values? If the later is true you probably could reduce the bits needed for each data point. Another thing is re-occuring patterns that are needed in all charts - could you factor those out in a separate object used by all charts hence only instantiated once?