What's the memory limit in WinXP when getting a callback from a C++ DLL in C#?

Question

I have a C# application that uses an unmanaged C++ DLL. I've found a crash that only happens in WinXP (not Win7) when the memory I'm passing back from the C++ DLL is too big.

The basic flow is that C# starts an operation in the C++ DLL by calling a start function, in which it provides a callback. The C++ DLL then performs the operation and dumps logging information into a text buffer. When the operation is completed the C++ DLL calls the callback and passes the text buffer as a parameter:

C++:

typedef void (CALLBACK *onfilecallbackfunc_t)(LPCWSTR);
DLL_API void NWAperture_SetOnFileCallback(onfilecallbackfunc_t p_pCallback);

l_pFileCallback(_wstringCapture.c_str());

C#:

public delegate void FileCallback([MarshalAs(UnmanagedType.LPWStr)] string buffer);
public static extern void SetOnFileCallback(FileCallback fileCallback);

private void OnFile(string buffer);

This works fine in Win7, but in WinXP if the buffer gets too big it crashes. I'm not sure of the exact size that causes this but I've put an 8MB limit on it and the crash has disappeared.

Does anyone know of a limit on the amount of memory that can be transferred between C++ and C# like this in WinXP? Or have I completely misunderstood this problem and there's a more logical explanation?

Update: I should have been more specific - this occurs on the same PC with WinXP and Win7 dual boot, both 32-bit OS.

Answer 1

So it eventually turned out I was being an idiot. In order to make the log large but speed the testing up I was clicking on the cancel button, which called a function in the C++ DLL that stopped execution and called the callback function with an 'abort' error and whatever log had already been recorded. But when I did this the execution didn't stop immediately, so when the callback with the log was in progress the C++ code could attempt to add to the log. This caused the instability I was seeing.

I fixed it by using a critical section around the log.

Answer 2

You can run out of contiguous memory way before you actually run out of RAM. This is the one big down side of using Array buffers. LinkedLists (or arrays using chunking) help mitigate this issue, because the space you need doesn't have to be contiguous.

So unless your application is using 2+ GB of RAM, than your issue is more likely memory fragmentation than anything else.

Windows 7 is probably managing the RAM differently than Windows XP, that's probably why you're not seeing the problem there. But push more data through and I'm sure you'll run into the same issue there.

You can setup perfmon to track/log memory usage of your system and task manager to track your application.