How to merge 2 string array in Delphi

Question

I have 2 or more dynamic string array that fill with some huge data , i want to merge this 2 array to one array , i know i can do it with a for loop like this :

var
  Arr1, Arr2, MergedArr: Array of string;
  I: Integer;
begin
  // Arr1:= 5000000 records
  // Arr2:= 5000000 records

  // Fill MergedArr by Arr1
  MergedArr:= Arr1;

  // Set length of MergedArr to length of ( Arra1 + Arr2 )+ 2
  SetLength(MergedArr, High(Arr1)+ High(Arr2)+2);

  // Add Arr2 to MergedArr
  for I := Low(Arr2)+1 to High(Arr2)+1 do
    MergedArr[High(Arr1)+ i]:= Arr2[i-1];
end;

but it is slow on huge data , is there faster way like copy array memory data ?

Answer 1

First of all string is special, so it should be treated specially: Don't try outsmarting the compiler, keep your code unchanged. String is special because it's reference counted. Every time you copy a string from one place to an other it's reference count is incremented. When the reference count reaches 0, the string is destroyed. Your code plays nice because it lets the compiler know what you're doing, and in turn the compiler gets the chance to properly increment all reference counts.

Sure, you can play all sorts of tricks as suggested in the comments to gabr's answer, like filling the old arrays with zero's so the reference count in the new array remains valid, but you can't do that if you actually need the old arrays as well. And this is a bit of a hack (albeit one that will probably be valid for the foreseeable future). (and to be noted, I actually like this hack).

Anyway, and this is the important part of my answer, your code is most likely not slow in the copying of the strings from one array to the other, it's most likely slowly somewhere else. Here's a short console application that creates two arrays, each with 5M random strings, then merges the two arrays into a third and displays the time it took to create the merge. Merging only takes about 300 milliseconds on my machine. Filling the array takes much longer, but I'm not timing that:

program Project26;

{$APPTYPE CONSOLE}

uses SysUtils, Windows;

var a, b, c: array of string;
    i: Integer;

    Freq: Int64;
    Start, Stop: Int64;
    Ticks: Cardinal;

const count = 5000000;

begin
  SetLength(a,count);
  SetLength(b,count);
  for i:=0 to count-1 do
  begin
    a[i] := IntToStr(Random(1));
    b[i] := IntToStr(Random(1));
  end;

  WriteLn('Moving');

  QueryPerformanceFrequency(Freq);
  QueryPerformanceCounter(Start);

  SetLength(c, Length(a) + Length(b));
  for i:=0 to High(a) do
    c[i] := a[i];
  for i:=0 to High(b) do
    c[i+Length(a)] := b[i];

  QueryPerformanceCounter(Stop);
  WriteLn((Stop - Start) div (Freq div 1000), ' milliseconds');
  ReadLn;

end.