FastMM4 says "The block footer has been corrupted"

Question

I have this function in our Delphi 7 application, which worked very well until I included FastMM4 v4.99 into the project. Once included, FastMM4 raised the following error message: "FastMM has detected an error during a FreeMem operation. The block footer has been corrupted." The execution halts in the FreeMem line.

function BinaryFieldToArrayOfWord( aBinaryField   : TVarBytesField;
                                   out aArrValues : TArrWord ) : Boolean;
var
  p : Pointer;
begin
  if not aBinaryField.IsBlob then
  begin
    GetMem( p, aBinaryField.DataSize );     
    try
      if aBinaryField.GetData( p ) then         
      begin
        // do something
      end;
    finally
      FreeMem( p, aBinaryField.DataSize );
    end;
  end; // if
end;

First I thought that there must be a bug in the function, but it is practically the same as this TField.GetData method example in the Delphi 7 help:

{ Retrieve the "raw" data from Field1 }
with Field1 do
begin
  if not IsBlob  { this does not work for BLOB fields }
  begin
    { Allocate space }
    GetMem(MyBuffer, DataSize);
    try
      if not GetData(MyBuffer) then
         MessageDlg(DisplayName + ' is NULL', mtInformation, [mbOK], 0)
      else 
         { Do something with the data };
    finally
      { Free the space }
      FreeMem(MyBuffer, DataSize);
    end;
  end;
end;

I found on the internet that the above error message is often because there is not enough space for the data. So I increased the size of the memory block and the error message vanished and the function worked as expected. After some experiment, I figured out that 2 bytes is required and enough:

    GetMem( p, aBinaryField.DataSize + 2 );

    FreeMem( p, aBinaryField.DataSize + 2 );

Although it solved my problem, I am not completely relaxed with this solution as I don't know its background. It would be nice to know the reason of this message. Does FastMM4 needs additional 2 bytes for its own footer?

UPDATE: After spending a day with debugging, I am now believe that there is a bug in the Delphi 7 TField.GetData method. It writes 2 zero bytes beyond the allocated memory block. I tried it with and without FastMM4, and it overwrites in both cases (so it is not a FastMM4 error). I also tried with typecasting the field as TVarBytesField, no difference. Here is the code I used with detailed comments, containing the results. My only remaining question: have they corrected this bug in later Delphis?

  procedure TfrmMain_PBC_TH.btnDEBUGClick(Sender: TObject);
  type
    TArrBytes = array of Byte;  
  var
    p       : Pointer;
    qryTest : TADOQuery;
  begin
    qryTest := TADOQuery.Create( Application );
    try
      // The type of the TQM_BinaryData.BinData column in the MSSQL database is a       varbinary(7900). 
      // Load the #168 binary data record. It contains exactly 7900 bytes, and the value of each byte is 255
      qryTest.Connection := MainConn;
      qryTest.SQL.Add('SELECT [BinData] FROM [TQM_BinaryData] WHERE [Id] = 168');
      qryTest.Open;

      // Allocate the memory block for this.
      GetMem( p, qryTest.FieldByName('BinData').DataSize );
      // DataSize is 7902 because all TVarBytesFields have 2 byte prefix in Delphi, containing the data length.
      // So the size of the allocated memory block is 7902 bytes - we are correct so far.
      try
        // Values of the first four bytes beyond the end of the memory block (memory thrash at this point) before GetData:
        // TArrBytes(p)[7902] = 96
        // TArrBytes(p)[7903] = 197
        // TArrBytes(p)[7904] = 219
        // TArrBytes(p)[7905] = 43

        // Critical point: get the data from the field with the Delphi GetData method
        qryTest.FieldByName('BinData').GetData( p );

        // Values after GetData:
        // TArrBytes(p)[0]    = 220    TArrBytes(p)[0] and TArrBytes(p)[1] contains the length of the binary data
        // TArrBytes(p)[1]    = 30     it is correct as 30 * 256 + 220 = 7900
        // TArrBytes(p)[2]    = 255    actual data starts
        // TArrBytes(p3[2]    = 255    
        // ...
        // TArrBytes(p)[7900] = 255
        // TArrBytes(p)[7901] = 255    actual data ends
        // TArrBytes(p)[7902] = 0      changed from 96!
        // TArrBytes(p)[7903] = 0      changed from 197!
        // TArrBytes(p)[7904] = 219    no change
        // TArrBytes(p)[7905] = 43     no change
      finally
        // Here FastMM4 throws the block footer corrupt error because GetData modified the 2 bytes after the allocated memory block 
        FreeMem( p );
      end;

      qryTest.Close;
    finally
      qryTest.Free;
    end;
  end;

After adding a Data Breakpoint, the call stack is:

    @FillChar(???,???,???)
    TDataSet.DataConvert($7D599770,$12F448,$7D51F7F0,True)
    VarToBuffer
    TCustomADODataSet.GetFieldData($7D599770,$7D51F7F0,True)
    TField.GetData($7D51F7F0,True)
    TfrmMain_PBC_TH.btnDEBUGClick($7FF7A380)
    TControl.Click
    TButton.Click

Answer 1

FastMM allocates some memory at the end of the block that you allocate and writes known values there. Then, when you deallocate, FastMM checks that those values are as expected. If not, then the error that you see is raised because FastMM knows that your code is writing beyond the end of the memory block.

So, something inside the try/finally block is writing beyond the end of the block of memory. And that's why increasing its size removes the FastMM error. Without seeing that code, we cannot tell what exactly is wrong and you will need to do some debugging to solve the problem.

You are quite right to be concerned by your "solution". Trial and error is never a reasonable way to program. You must find out why your program is writing beyond the end of the block.

One way to do that is to set a data breakpoint for the address immediately beyond the end of this block. That would force the debugger to break on the code that writes beyond the end of the block.

As an aside, you don't need to pass the second parameter to FreeMem. Doing so makes your code harder to maintain and serves absolutely no purpose. Pass just the pointer to FreeMem.