Using a large static array in C# (Silverlight on Windows Phone 7)

Question

I have a question that's so simple I cannot believe I can't answer it myself. But, there you go.

I have a large-ish static list (of cities, latitudes and longitudes) that I want to use in my Windows Phone 7 Silverlight application. There are around 10,000 of them. I'd like to embed this data statically in my application and access it in an array (I need to cycle through the whole list in code pretty regularly).

What is going to be my most effective means of storing this? I'm a bit of an old school sort, so I reckoned the fastest way to do it would be:

public struct City
{
    public string name;
    public double lat;
    public double lon;
};

and then...

private City[] cc = new City[10000];

public CityDists()
{
    cc[2].name = "Lae, Papua New Guinea"; cc[2].lat = 123; cc[2].lon = 123;
    cc[3].name = "Rabaul, Papua New Guinea"; cc[3].lat = 123; cc[3].lon = 123;
    cc[4].name = "Angmagssalik, Greenland"; cc[4].lat = 123; cc[4].lon = 123;
    cc[5].name = "Angissoq, Greenland"; cc[5].lat = 123; cc[5].lon = 123;
...

However, this bums out with an "out of memory" error before the code actually gets to run (I'm assuming the code itself ended up being too much to load into memory).

Everything I read online tells me to use an XML resource or file and then to deserialise that into instances of a class. But can that really be as fast as using a struct? Won't the XML take ages to parse?

I think I'm capable of writing the code here - I'm just not sure what the best approach is to start with. I'm interested in speed of load and (more importantly) run time access more than anything.

Any help very much appreciated - first question here so I hope I haven't done anything boneheaded.

Chris

Answer 1

10,000 structs shouldn't run out of memory, but just to make sure, I would first try turning your struct into a class such that it uses the heap instead of the stack. There is a strong possibility that doing that will fix your out of memory errors.

An XML file stored in isolated storage might be a good way to go if your data is going to be updated even every once in a while. You could pull the cities from a web service and serialize those classes to the Application Store in isolated storage whenever they get updated.

Also, I notice in the code samples that the cc array is not declared static. If you have a few instances of CityDists, then that could also be bogging down memory as the array is getting re-created every time a new CityDists class is created. Try declaring your array as static and initializing it in the static constructor:

private static City[] cc = new City[10000];

static CityDists()
{
    cc[2].name = "Lae, Papua New Guinea"; cc[2].lat = 123; cc[2].lon = 123;
    cc[3].name = "Rabaul, Papua New Guinea"; cc[3].lat = 123; cc[3].lon = 123;
    cc[4].name = "Angmagssalik, Greenland"; cc[4].lat = 123; cc[4].lon = 123;
    cc[5].name = "Angissoq, Greenland"; cc[5].lat = 123; cc[5].lon = 123;
...

Answer 2

If loading an xml doc from the xap works for you..

Here's a project I posted demonstrating loading of an xml doc from the XAP via XDocument/LINQ and databinding to a listbox for reference.

binding a Linq datasource to a listbox

Answer 3

If you want to avoid the XML parsing and memory overhead, you could use a plain text file for storing your data and use the .Net string tokenizer functions to parse the entries e.g. use String.Split()

You could also load the file partially to keep memory consumption low. For example, you load only k out of n lines of the file. In case you need to access a record that is outside the currently loaded k segments, load the appropriate k segments. You could either do it the old school way or even use the fancy Serialization stuff from .Net