stack overflow error in delphi

Question

I have a procedure which calls several functions:

procedure TForm1.Button1Click(Sender: TObject);
var
  rawData: TRawData;
  rawInts: TRawInts;
  processedData: TProcessedData;
begin
  rawData := getRawData();
  rawInts := getRawInts(rawData);
  processedData := getProcessedData(rawInts);
end;

The data types are defined like this:

TRawData = array[0..131069] of Byte;
TRawInts = array[0..65534] of LongInt;
TProcessedData = array[0..65534] of Double;

running the program with just:

rawData := getRawData();
rawInts := getRawInts(rawData);

Works totally fine. However, when I try to run:

getProcessedData(rawInts)

I get a stackoverflow error. I don't see why this is. The function code for getProcessedData is very simple:

function getProcessedData(rawInts : TRawInts) : TProcessedData;
var
  i: Integer;
  tempData: TProcessedData;
  scaleFactor: Double;
begin
  scaleFactor := 0.01;

  for i := 0 to 65534 do
    tempData[i] := rawInts[i] * scaleFactor;

  Result := tempData;
end;

Why is this causing an error ?

Answer 1

The default maximum stack size for a thread is 1 MB. The three local variables of Button1Click total 131,070 + 65,535 * 4 + 65,535 * 8 = 917,490 bytes. When you call getProcessedData, you pass the parameter by value, which means that the function makes a local copy of the parameter on the stack. That adds SizeOf(TRawInts) = 262,140 bytes to bring the stack to at least 1,179,630 bytes, or about 1.1 MB. There's your stack overflow.

You can reduce the stack use by passing the TRawInts array by reference instead. Then the function won't make its own copy. Zdravko's answer suggests using var, but since the function has no need to modify the passed-in array, you should use const instead.

function getProcessedData(const rawInts: TRawInts): TProcessedData;

Naively, we might expect the tempData and Result variables in getProcessedData to occupy additional stack space, but in reality, they probably won't. First, large return types typically result in the compiler changing the function signature, so it would act more like your function were declared with a var parameter instead of a return value:

procedure getProcessedData(rawInts: TRawInts; var Result: TProcessedData);

Then the call is transformed accordingly:

getProcessedData(rawInts, processedData);

Thus, Result doesn't take up any more stack space because it's really just an alias for the variable in the caller's frame.

Furthermore, sometimes the compiler recognizes that an assignment at the end of your function, like Result := tempData, means that tempData doesn't really need any space of its own. Instead, the compiler may treat your function as though you had been writing directly into Result all along:

begin
  scaleFactor := 0.01;

  for i := 0 to 65534 do
    Result[i] := rawInts[i] * scaleFactor;
end;

However, it's best not to count on the compiler to make those sorts of memory-saving changes. Instead, it's better not to lean so heavily on the stack in the first place. To do that, you can use dynamic arrays. Those will move the large amounts of memory out of the stack and into the heap, which is the part of memory used for dynamic allocation. Start by changing the definitions of your array types:

type
  TRawData = array of Byte;
  TRawInts = array of Integer;
  TProcessedData = array of Double;

Then, in your functions that return those types, use SetLength to assign the length of each array. For example, the function we've seen already might go like this:

function getProcessedData(const rawInts: TRawInts): TProcessedData;
var
  i: Integer;
  scaleFactor: Double;
begin
  Assert(Length(rawInts) = 65535);
  SetLength(Result, Length(rawInts));

  scaleFactor := 0.01;

  for i := 0 to High(rawInts) do
    Result[i] := rawInts[i] * scaleFactor;
end;