Stack runtime check failure with sqrt intrinsic in VS2012

Question

While debugging some crash, I've come across some code which simplifies down to the following case:

#include <cmath>
#pragma intrinsic (sqrt) 

class MyClass
{
public:
  MyClass() { m[0] = 0; }
  double& x() { return m[0]; }
private:
  double m[1];
};
void function()
{
  MyClass obj;
  obj.x() = -sqrt(2.0);
}

int main()
{
  function();
  return 0;
}

When built in Debug|Win32 with VS2012 (Pro Version 11.0.61030.00 Update 4, and Express for Windows Desktop Version 11.0.61030.00 Update 4), the code triggers run-time check errors at the end of the function execution, which show up as either (in a random fashion):

Run-Time Check Failure #2 - Stack around the variable 'obj' was corrupted.

or

A buffer overrun has occurred in Test.exe which has corrupted the program's internal state. Press Break to debug the program or Continue to terminate the program.

I understand that this usually means some sort of buffer overrun/underrun for objects on the stack. Perhaps I'm overlooking something, but I can't see anywhere in this C++ code where such a buffer overrun could occur. After playing around with various tweaks to the code and stepping through the generated assembly code of the function (see "details" section below), I'd be tempted to say it looks like a bug in Visual Studio 2012, but perhaps I'm just in too deep and missing something.

Are there intrinsic function usage requirements or other C++ standard requirements that this code does not meet, which could explain this behaviour?

If not, is disabling function intrinsic the only way to obtain correct run-time check behaviour (other than workaround such as 0-sqrt noted below which could easily get lost)?

The details

Playing around the code, I've noted that the run-time check errors go away when I disable the sqrt intrinsic by commenting out the #pragma line.

Otherwise with the sqrt intrinsic pragma (or the /Oi compiler option) :

• Using a setter such as obj.setx(double x) { m[0] = x; }, not surprisingly also generates the run-time check errors.
• Replacing obj.x() = -sqrt(2.0) with obj.x() = +sqrt(2.0) or obj.x() = 0.0-sqrt(2.0) to my surprise does not generate the run-time check errors.
• Similarly replacing obj.x() = -sqrt(2.0) with obj.x() = -1.4142135623730951; does not generate the run-time check error.
• Replacing the member double m[1]; with double m; (along with m[0] accesses) only seem to generate the "Run-Time Check Failure #2" error (even with obj.x() = -sqrt(2.0)), and sometimes runs fine.
• Declaring obj as a static instance, or allocating it on the heap does not generate the run-time check errors.
• Setting compiler warnings to level 4 does not produce any warnings.
• Compiling the same code with VS2005 Pro or VS2010 Express does not generate the run-time check errors.
• For what it's worth, I've noted the problem on a Windows 7 (with Intel Xeon CPU) and a Windows 8.1 machine (with Intel Core i7 CPU).

Then I went on to look at the generated assembly code. For the purpose of illustration, I will refer to "the failing version" as the one obtained from the code provided above, whereas I've generated a "working version" by simply commenting the #pragma intrinsic (sqrt) line. A side-by-side diff view of the resulting generated assembly code is shown below with the "failing version" on the left, and the "working version" on the right:

First I've noted that the _RTC_CheckStackVars call is responsible for the "Run-Time Check Failure #2" errors and checks in particular whenever the magic cookies 0xCCCCCCCC are still intact around the obj object on the stack (which happens to be starting at an offset of -20 bytes relative to the original value of ESP). In the following screenshots, I've highlighted the object location in green and the magic cookie location in red. At the start of the function in the "working version" this is what it looks like:

then later right before the call to _RTC_CheckStackVars:

Now in the "failing version", the preamble include an additional (line 3415)

and         esp,0FFFFFFF8h

which essentially makes obj aligned on a 8 byte boundary. Specifically, whenever the function is called with an initial value of ESP that ends with a 0 or 8 nibble, the obj is stored starting at an offset of -24 bytes relative to the initial value of ESP. The problem is that the _RTC_CheckStackVars still looks for those 0xCCCCCCCC magic cookies at those same locations relative to the original ESP value as in the "working version" depicted above (ie. offsets of -24 and -12 bytes). In this case, obj's first 4 bytes actually overlaps one of the magic cookie location. This is shown in the screenshots below at the start of the "failing version":

then later right before the call to _RTC_CheckStackVars:

We can note in passing the the actual data which corresponds to obj.m[0] is identical between the "working version" and the "failing version" ("cd 3b 7f 66 9e a0 f6 bf", or the expected value of -1.4142135623730951 when interpreted as a double).

Incidentally, the _RTC_CheckStackVars checks actually passes whenever the initial value of ESP ends with a 4 or C nibble (in which case obj starts at a -20 bytes offset, just like in the "working version").

After the _RTC_CheckStackVars checks complete (assuming it passes), there is an additional check that the restored value of ESP corresponds to the original value. This check, when it fails, is responsible for the "A buffer overrun has occurred in ..." message.

In the "working version", the original ESP is copied to EBP early in the preamble (line 3415) and it's this value which is used to compute the checksum by xoring with a ___security_cookie (line 3425). In the "failing version", the checksum computation is based on ESP (line 3425) after ESP has been decremented by 12 while pushing some registers (lines 3417-3419), but the corresponding check with the restored ESP is done at the same point where those registers have been restored.

So, in short and unless I didn't get this right, it looks like the "working version" follows standard textbook and tutorials on stack handling, whereas the "failing version" messes up the run-time checks.

P.S.: "Debug build" refers to the standard set of compiler options of the "Debug" config from the "Win32 Console Application" new project template.

Answer 1

As pointed out by Hans in comments, the issue can no longer be reproduced with the Visual Studio 2013. Similarly, the official answer on Microsoft connect bug report is:

we are unable to reproduce it with VS2013 Update 4 RTM. The product team itself no longer directly accepting feedback for Microsoft Visual Studio 2012 and earlier products. You can get support for issues with Visual Studio 2012 and earlier by visiting one of the resources in the link below: http://www.visualstudio.com/support/support-overview-vs

So, given that the problem is triggered only on VS2012 with function intrinsics (/Oi compiler option), runtime-checks (either /RTCs or /RTC1 compiler option) and usage of unary minus operator, getting rid of any one (or more) of those conditions should work around the problem.

Thus, it seems the available options are:

1. Upgrade to the latest Visual Studio (if your project permits)
2. Disable runtime checks for the affected functions by surrounding them with #pragma runtime_check such as in the following sample:

    #pragma runtime_check ("s", off)
    void function()
    {
      MyClass obj;
      obj.x() = -sqrt(2.0);
    }
    #pragma runtime_check ("s", restore)

3. Disable intrinsics by removing the #pragma intrinsics (sqrt) line, and adding #pragma function (sqrt) (see msdn for more info).
   If intrinsics have been activated for all files through the "Enable Intrinsic Functions" project property (/Oi compiler option), you would need to deactivate that project property. You can then enable intrinsics on a piece-by-piece basis for specific functions while checking that they are not affected by the bug (with #pragma intrinsics directives for each required intrinsic function).
4. Tweak the code using workarounds such as 0-sqrt(2.0), -1*sqrt(2.0) (which remove the unary minus operator) in an attempt to fool the compiler into using a different code generation path. Note that this is very likely to break with seemingly minor code changes.