C# memory usage for creating objects in a for loop

Question

I have a complex database conversion console app that reads from an old database, does a bunch of things, and puts into the new database.

I'm having an escalating memory problem where my mem usage (as monitored in task manager) constantly climbs and eventually slows down the process to a halt.

I've boiled it down to the simplest possible test POC to try and understand what's going on.

for (int i = 0; i < 100000; i++)
{
  TestObj testc = new TestObj
    {
      myTest = "testing asdf"
    };
}
public class TestObj
{
    public string myTest;
}

My thought was that each testc that is created in the loop wouldn't survive past the end of the iteration, but the way the memory is tracking it seems like the application is holding on to every instance of testc.

I've done a good amount of research and experimentation but I feel like there is something I'm missing here. Shouldn't I be able to run this and have memory utilization stay rather constant?

Answer 1

Life will be a little easier if you use a struct instead...

for (int i = 0; i < 100000; i++)
{
    TestObj testc = new TestObj
    {
        myTest = "testing asdf"
    };
}

public struct TestObj
{
    public string myTest;
}

This still requires the allocation of the string but the struct won't survive. It depends on what your class really looks like, if you have a lot of value types in it this will help immensely. If you have a bunch of string/reference values your still in trouble.

Otherwise you can do something like the following:

for (int i = 0; i < 100000; i++)
{
    // do your work...

    // then every 1k cycles, see if we have > 100mb allocated
    // and force the GC to free the memory
    if(i % 1000 == 0 && GC.GetTotalMemory(false) > 100000000)
        GC.Collect();
}

Note: This is an ugly 'hacky' sort of thing to do; however, sometimes it's the quickest solution to the problem.

Update

Addtionally you need to make sure you are not hitting the LOH (Large object heap) as this can be a source of memory contention. As a general rule keep strings, byte[], ect, under 85kb. This means that strings need to me less than 42k characters in length.

Answer 2

You're missing one key thing about garbage collection: The GC won't run until it needs to. So yes, even though it's nice enough to clean up after you, it still won't do it until more memory is needed.