StackOverFlowException in WPF when call method from C++ library

Question

I use C++ library in WPF. It's SDK for Magnetic Stripe Reader/Writer. When I call one of it's methods in WPF, I receive StackOverFlowException after 10 seconds. Method called from button click event.

[DllImport("MSR_API.dll")]
static extern bool MSR_InitComm(string portname, UInt32 baud);

This method connects to Magnetic Stripe Reader device. First I tested this method on Windows Forms application, and everything was great. But when I started to write WPF app with this library, I receive StackOverFlowException everytime. What could be the reason of this "feature"?

Answer 1

Getting the [DllImport] declaration wrong is a standard cause. The CallingConvention property is very important. Getting it wrong causes the stack to get imbalanced, ultimately causing SO if you call it often enough. There's an MDA for that, make sure you didn't turn off PinvokeStackImbalance. Using Debug + Windows + Registers is another way to diagnose it, the ESP register value must be the same before and after the call.

CallingConvention.Cdecl is often required for C or C++ code unless that code was explicitly written with the __stdcall keyword.

---

Okay, the Embarcadero link suggests another reason for the exception. Borland libraries traditionally enabled FPU exceptions. That's grossly incompatible with .NET code. Especially WPF because it heavily uses doubles for control sizes and positions. An FPU stack overflow is a bit odd, you more typically have trouble with NaN values.

If you don't have the source code of the library then you don't have many attractive options to fix the problem. One thing you can try is to throw and catch an exception after the first call to the library. The .NET exception handling plumbing resets the FPU control word. Like this:

bool ok = MSR_InitComm("COM1", 9600);
try {
    throw new Exception("Fpu reset intended");
}
catch (Exception) {
}