malloc() returning address that I cannot access

Question

I have a C++ program that calls some C routines that are generated by Flex / Bison.

When I target a Windows 8.1 64-bit platform, I hit the following exception at runtime:

Unhandled exception at 0x0007FFFA70F2C39 (libapp.dll) in application.exe: 0xC0000005: 
Access violation writing location 0x000000005A818118.

I traced this exception to the following piece of code:

YY_BUFFER_STATE yy_create_buffer( FILE *file, int size )
{
   YY_BUFFER_STATE b;
   b = (YY_BUFFER_STATE) yy_flex_alloc( sizeof( struct yy_buffer_state ) );
   if ( ! b )
      YY_FATAL_ERROR( "out of dynamic memory in yy_create_buffer()" );
   b->yy_buf_size = size; // This access is what throws the exception
}

For reference, elsewhere in the code (also generated by Flex / Bison), we have:

typedef struct yy_buffer_state *YY_BUFFER_STATE;
struct yy_buffer_state
{
   FILE *yy_input_file;
   char *yy_ch_buf;
   char *yy_buf_pos;
   yy_size_t yy_buf_size;
   // ... other fields omitted,
   // total struct size is 56 bytes
}

static void *yy_flex_alloc( yy_size_t size )
{
   return (void *) malloc( size );
}

I traced back to the malloc call and observed that malloc itself is returning the address 0x000000005A818118. I also checked errno, but it is not set after the call to malloc.

My question is: why does malloc give me an address that I don't have access to, and how can I make it give me a correct address?

Note: I only observe this behavior in Windows 8.1 64-bit. It passes with other 32-bit Windows variants, as well as Windows 7 32-bit.

Compilation information: I am compiling this on a 64-bit Windows 8.1 machine using Visual Studio 2012.

If it helps, here is the disassembled code:

// b = (YY_BUFFER_STATE) yy_flex_alloc( ... )
0007FFFA75E2C12  call   yy_flex_alloc (07FFFA75E3070h)
0007FFFA75E2C17  mov    qword ptr [b],rax
// if ( ! b ) YY_FATAL_ERROR( ... )
0007FFFA75E2C1C  cmp    qword ptr [b],0
0007FFFA75E2C22  jne    yy_create_buffer+30h (07FFFA75E2C30h)
0007FFFA75E2C24  lea    rcx,[yy_chk+58h (07FFFA7646A28h)]
0007FFFA75E2C2B  call   yy_fatal_error (07FFFA75E3770h)
// b->yy_buf_size = size
0007FFFA75E2C30  mov    rax,qword ptr [b]
0007FFFA75E2C35  mov    ecx,dword ptr [size]
0007FFFA75E2C39  mov    dword ptr [rax+18h],ecx

Thanks!

Answer 1

The real answer is:

When you are compiling flex-generated .c source in Visual Studio, it doesn't include stdlib.h (where malloc defined as returning void*) and Visual Studio takes some own definition, where malloc returns int. (I think it's for some kind of compatibility)

Visual studio prints:

'warning C4013: 'malloc' undefined; assuming extern returning int' sizeof(int)==4, but values in pointers on x64 systems often exceed 4 bytes

So your pointer just cut to low 4 bytes.

It seems this problem appears only in x64 bits visual studio in .c files.

So, solution will be - just include stdlib.h by yourself, or define some macros, which will lead in flex-generated source to including stdlib.h.