Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

C# compiler (csc.exe) memory overflow compiling nested types and Linq

I found one unpleasant behavior developing with Visual Studio. It was hanging my machine while compiling C#.

I have reduced behavior to the next minimal source code

using System.Collections.Generic;
using System.Linq;

namespace memoryOverflowCsharpCompiler {

    class SomeType { public decimal x; }

    class TypeWrapper : Dictionary<int,
                        Dictionary<int,
                        Dictionary<int, SomeType [] []>>> {

        public decimal minimumX() {
            return base.Values.Min(a =>
                      a.Values.Min(b =>
                      b.Values.Min(c =>
                      c       .Sum(d =>
                      d       .Sum(e => e.x)))));
        }
    }
}

Compiling with

PROMPT> csc source.cs

    *** BANG! overflow memory usage (up to ~3G)

PROMPT> csc /?
Microsoft (R) Visual C# Compiler version 12.0.30501.0
Copyright (C) Microsoft Corporation. All rights reserved.
...

(using Windows 8.1 Pro N x64; csc compiler process is running with 32bits)

sightly modifications don't produce this behavior (eg. changing decimal by int, reducing one nested level, ...), performing a big Select then reducing, works fine

Explicit workaround:

            return base.Values.SelectMany(a =>
                      a.Values.SelectMany(b =>
                      b.Values.Select    (c =>
                      c.       Sum       (d =>
                      d.       Sum       (e => e.x))))).Min();

Although this explicit workaround exists, it not guaranteed that this behavior will not occur again.

What's wrong?

Thank you!

like image 554
josejuan Avatar asked Dec 17 '14 10:12

josejuan


1 Answers

It seems that generic type resolution fails in that case. Changing from decimal to int works by chance. If you increase nesting level, than you'll see that it also fails for int. On my x64 machine this code compiles for both int and decimal and uses around 2.5GB of memory, but increasing nesting level results in overflow when memory usage growing to aroung 4GB.

Specifying type argument explicitly allows to compile code:

class TypeWrapper : Dictionary<int, Dictionary<int, Dictionary<int, Dictionary<int, SomeType[][]>>>>
{
    public decimal minimumX()
    {
        return base.Values
            .Min<Dictionary<int, Dictionary<int, Dictionary<int, SomeType[][]>>>, decimal>(a => a.Values
                .Min<Dictionary<int, Dictionary<int, SomeType[][]>>, decimal>(b => b.Values
                    .Min<Dictionary<int, SomeType[][]>, decimal>(c => c.Values
                        .Min(d => d
                            .Sum(e => e.Sum(f => f.x))
                        )
                    )
                )
            );
    }
}

Also compiler works when you reduce nesting by introducing local variable:

class TypeWrapper : Dictionary<int, Dictionary<int, Dictionary<int, Dictionary<int, SomeType[][]>>>>
{
    public decimal minimumX()
    {
        Func<Dictionary<int, SomeType[][]>, decimal> inner = (Dictionary<int, SomeType[][]> c) => c.Values
                        .Min(d => d
                            .Sum(e => e.Sum(f => f.x))
                        );

        return base.Values
            .Min(a => a.Values
                .Min(b => b.Values
                    .Min(inner)
                )
            );
    }
}
like image 144
Krzysztof Avatar answered Oct 30 '22 16:10

Krzysztof