C# application with unmanaged DLL freezes whole system

Question

I'm currently writing a software in Visual Studio 2012 for communication with RFID-cards. I got a DLL written in Delphi to handle the communication with the card reader.

The problem is: My software is running fine on machines, that have VS2012 installed. On other systems it freezes itself or the whole system. I tried it on Win XP / 7 / 8 with x32 and x64 configuration. I'm using .NET 4.0.

After connecting to the reader, the software starts a backgroundWorker, which polls (at 200ms rate) the reader with a command to inventory cards in the readers RF-field. The crash usally happens ca. 10 to 20 seconds after the reader connect. Here is the code:

[DllImport("tempConnect.dll", CallingConvention = CallingConvention.StdCall)]
 private static extern int inventory(int maxlen, [In] ref int count, 
                                         IntPtr UIDs, UInt32 HFOffTime);

public String getCardID()
    {
        if (isConnectet())
        {
            IntPtr UIDs = IntPtr.Zero;
            int len = 2 * 8;
            Byte[] zero = new Byte[len];
            UIDs = Marshal.AllocHGlobal(len);
            Thread.Sleep(50);
            Marshal.Copy(zero, 0, UIDs, len);
            int count = 0;
            int erg;
            String ret;
            try
            {
                erg = inventory(len, ref count, UIDs, 50);
            }
            catch (ExternalException) // this doesn't catch anything (iI have set <legacyCorruptedStateExceptionsPolicy enabled="true"/>)
            {
                return "\0";
            }
            finally
            {
                ret = Marshal.PtrToStringAnsi(UIDs, len);
                IntPtr rslt = LocalFree(UIDs);
                GC.Collect();
            }
            if (erg == 0)
                return ret;
            else
                return zero.ToString();
        }
        else
            return "\0";
    }

The DLL is written in Delphi, the code DLL command is:

function inventory (maxlen: Integer; var count: Integer; 
                  UIDs: PByteArray; HFOffTime: Cardinal = 50): Integer; STDCALL;

I think there may be a memory leak somewhere, but I have no idea how to find it...

---

EDIT:

I added some ideas (explicit GC.Collect(), try-catch-finally) to my code above, but it still doesnt work.

Here is the code, that calls getCardID():

The action, that runs every 200ms:

if (!bgw_inventory.IsBusy)
   bgw_inventory.RunWorkerAsync();

Async backgroundWorker does:

private void bgw_inventory_DoWork(object sender, DoWorkEventArgs e)
    {
            if (bgw_inventory.CancellationPending)
        {
            e.Cancel = true;
            return;
        }
        else
        {
            String UID = reader.getCardID();
            if (bgw_inventory.CancellationPending)
            {
                e.Cancel = true;
                return;
            }
            if (UID.Length == 16 && UID.IndexOf("\0") == -1)
            {
                setCardId(UID);
                if (!allCards.ContainsKey(UID))
                {
                    allCards.Add(UID, new Card(UID));
                }
                if (readCardActive || deActivateCardActive || activateCardActive)
                {
                    if (lastActionCard != UID)
                        actionCard = UID;
                    else
                        setWorkingStatus("OK", Color.FromArgb(203, 218, 138));
                }
            }
            else
            {
                setCardId("none");
                if (readCardActive || deActivateCardActive || activateCardActive)
                    setWorkingStatus("waiting for next card", Color.Yellow);
            }
        }
   }

---

EDIT

Till now I have made some little reworks (updates above) at the code. Now only the App. crashes with 0xC00000FD (Stack overflow) at "tempConnect.dll". This does not happen on Systems with VS2012 installed or if I use the DLL with native Delphi! Do anyone have any other ideas ?

---

EDIT

Now I made the DLL logging it's stacksize and found something weird: If it's called and polled from my C# Programm, the stacksize is changing continuously up and down. If i do the same from a natural Deplhi Program the stacksize is constant! So I'll do further investigations, but I have no really idea, what I have to search for...

Answer 1

I'm a little concerned about how're using that Marshal object. As you fear with the memory leak, it seems to be allocating memory quite often but I don't see it ever explicitly releasing it. The garbage collector should (operative word) be taking care of that, but you say yourself you have some unmanaged code in the mix. It is difficult with the posted information to tell where the unmanaged code begins.

Check out this question for some good techniques to finding memory leaks in .NET itself - this will give you a ton of information on how memory is being used in the managed end of your code (that is, the part you can directly control). Use the Windows Performance Monitor with breakpoints to keep an eye on the overall health of the system. If .NET appears to be behaving, but WPM is showing some sharp spikes, it's probably in the unmanaged code. You can't really control anything but your usage there, so it would probably be time to go back to the documentation at that point.