Why would this access violation occur with the /Og and /GL flags, with pass-by-reference?

Question

When (and only when) I compile my program with the /Og and /GL flag using the Windows Server 2003 DDK C++ compiler (it's fine on WDK 7.1 as well as Visual Studio 2010!), I get an access violation when I run this:

#include <algorithm>
#include <vector>

template<typename T> bool less(T a, T b) { return a < b; }

int main()
{
    std::vector<int> s;

    for (int i = 0; i < 13; i++)
        s.push_back(i);

    std::stable_sort(s.begin(), s.end(), &less<const int&>);
}

The access violation goes away when I change the last line to

    std::stable_sort(s.begin(), s.end(), &less<int>);

-- in other words, it goes away when I let my item get copied instead of merely referenced.

(I have no multithreading of any sort going on whatsoever.)

Why would something like this happen? Am I invoking some undefined behavior through passing by const &?

---

• Compiler flags:

    /Og /GL /MD /EHsc
  

• Linker flags: (none)

• INCLUDE environmental variable:

    C:\WinDDK\3790.1830\inc\crt
  

• LIB environmental variable:

    C:\WinDDK\3790.1830\lib\crt\I386;C:\WinDDK\3790.1830\lib\wxp\I386
  

• Operating system: Windows 7 x64

• Platform: 32-bit compilation gives error (64-bit runs correctly)

---

Edit:

I just tried it with the Windows XP DDK (that's C:\WinDDK\2600) and I got:

error LNK2001: unresolved external symbol
    "bool __cdecl less(int const &,int const &)" (?less@@YA_NABH0@Z)

but when I changed it from a template to a regular function, it magically worked with both compilers!

I'm suspecting this means that I've found a bug that happens while taking the address of a templated function, using the DDK compilers. Any ideas if this might be the case, or if it's a different corner case I don't know about?

Answer 1

I tried this with a Windows Server 2003 DDK SP1 installation (the non-SP1 DDK isn't readily available at the moment). This uses cl.exe version 13.10.4035 for 80x86. It appears to have the same problem you've found.

If you step through the code in a debugger (which is made a bit easier by following along with the .cod file generated using the /FAsc option) you'll find that the less<int const &>() function expects to be called with the pointers to the int values passed in eax and edx. However, the function that calls less<int const&>() (named _Insertion_sort_1<>()) calls it passing the pointers on the stack.

If you turn the templated less function into a non-templated function, it expects the parameters to be passed on the stack, so everyone is happy.

Of a bit more interest is what happens when you change less<const int&> to be less<int> instead. There's no crash, but nothing gets sorted either (of course, you'd need to change your program to start out with a non-sorted vector to actually see this effect). That's because when you change to less<int> the less function no longer dereferences any pointers - it expects the actual int values to be passed in registers (ecx and edx in this case). But no pointer dereference means no crash. However, the caller, _Insertion_sort_1, still passes the arguments on the stack, so the comparison being performed by less<int> has nothing to do with the values in the vector.

So that's what's happening, but I don't really know what the root cause is - as others have mentioned, it looks like a compiler bug related to the optimizations.

Since the bug has apparently been fixed, there's obviously no point in reporting it (the compiler in that version of the DDK corresponds to something close to VS 2003/VC 7.1).

By the way - I was unable to get your example to compile completely cleanly - to get it to build at all, I had to include the bufferoverflowu.lib to get the stack checking stuff to link, and even then the linker complained about "multiple '.rdata' sections found with different attributes". I seem to remember that being a warning that was safe to ignore, but I really don't remember. I don't think either of these has anything to do with the bug though.