Storing data in memory, which is a good way?

Question

I have a items.xml file, which is loaded into memory once the application starts. Each element in items.xml is "converted" to type Item (which has just few int & string properties), then added into a list of items. The item list should hold thousands instances of Item.

Storing data in objects (Item class in my example) is ok? Is there another way to store such data in memory?

Answer 1

First worry about getting the functionality right. Second, worry about performance. Profile the application and see where it hurts performance-wise. It may never be in your config. If, however, you suffer performance problems in this area, try to invent keys for either single or subsets of Items. This way you can populate one or several dictionaries with keys and Items (or lists of items) thereby reducing the access time to the Item in question.

Remember that these look-up dictionaries are "cheap" in that they only store references to your items rather than copies of the objects.

Answer 2

That is fine, and as long as you ensure you don't eat all the memory should be fine. I've done this for complex systems with larger numbers.

If your data is regularly accessed by something other than the index in the list, you might want to use a dictionary to index the item, for example:

Dictionary<string, Item> lookup =
    list.ToDictionary(x => x.Code);

Then you can access with:

Item item = lookup["abc012"];

Also; if many of the string values ate repeated you can save some apace by writing your own interner;

Dictionary<string,string> interner =
    new Dictionary<string,string>();
foreach(Item item in list) {
    string s, name = item.Name;
    if(interner.TryGetValue(name, out s))
        item.Name = s;
    else
        interner.Add(name, name);
}

Which will reduce memory use by only keeping and re-using unique strings