Delphi 7, strings problems

Question

In my project I have a problem with strings "out of memory exception", MM is not used. The problem shows when lengths of the string goes to 2 300,000 symbols. Despite the fact that there is enough memory and in the same part of code I can create a sting with 100,000,000 characters.

Google did not help, I can't disassemble it (have no skills), so I decided to create a minimal test example, where I can get out of memory exception on string less then 2 000 000 000 symbols. I could not create such example, but I created something stranger:

program Project2;
{$APPTYPE CONSOLE}
uses
   SysUtils;

var s : string;
    k : integer;

function b : string;
begin
 result := 'f';
end;

procedure c;
var ss : string;
begin
  s := s + '{' +  b + '}';
  ss :=  'a';

  if k mod 100001 = 0 then
  begin
     // ss[1] := 'd';    // uncoment me
     write(k mod 10);
  end;

  inc(k);
end;

begin
  while true do c;
end.

This code works fine. It just adds something to a global string with some extra operations. The thing is if you uncomment marked string it will slow down significantly (with or without optimization). Considering that this assign value once in 100,001 iterations, it must not slow down.

Questions:

1. How do default strings in Delphi work?

2. How to avoid slow down?

3. How to avoid out of memory?

P.S. If I include FastMM into the main project the error disappears p.p.s The example with the string uncommented sends my Windows 7 to BSOD in 3 minutes (from user mode).

Answer 1

Allocating strings by doing

s := s + '{' +  b + '}';

in a long running loop will simply fragment your memory. You may well have sufficient memory for the string but that's not enough. You need the memory to be contiguous but your allocation pattern will make that hard.

Solve the problem by pre-allocating the string to its final desired length with a call to SetLength.

Answer 2

1. How do default strings in Delphi work?

A new string is allocated each time it is affected (via a :=).

That is,

s := s + '{' +  b + '}';

will allocate a string for s + '{' + b + '}';, then copy it to variable s.

Each time you run this line, you have one memory allocation, and one memory release. This can be slow, even with FastMM4. But with the older MM, it can be dead slow.

2. How to avoid slow down?

If you are in old Delphi, with the "Borland" memory manager, allocation and reallocation is very slow. And it will fragment the memory a lot.

The ss[1] := 'd' is certainly very slow due to this memory fragmentation, and the fact that the Borland Memory Manager has to make some slow clean-up for the memory allocation of this line.

Change the line with:

var ss: string[1];

And it won't slow down any more, since the shortstring will be allocated on the stack, and the heap won't be used.

So to avoid slow down:

• Use a modern Memory Manager, like FastMM4;
• Use a TStringBuilder-like class, or a good old TMemoryStream in which you append your data: you'll have much less memory reallocation, so it will be much faster.

3. How to avoid out of memory?

Out of memory error comes from memory fragmentation.

So the two solutions of the previous question will fix this.