Is this Delphi dynamic array behaviour expected

Question

The question is - how dynamic arrays are managed internally by Delphi when they are set as a class member? Are they copied or passed by reference? Delphi 10.3.3 used.

The UpdateArray method deletes the first element from the array. But the array length stays 2. The UpdateArrayWithParam method also deletes the first element from the array. But the array length is correctly reduced to 1.

Here is a code sample:

interface

type
  TSomeRec = record
      Name: string;
  end;
  TSomeRecArray = array of TSomeRec;

  TSomeRecUpdate = class
    Arr: TSomeRecArray;
    procedure UpdateArray;
    procedure UpdateArrayWithParam(var ParamArray: TSomeRecArray);
  end;

implementation

procedure TSomeRecUpdate.UpdateArray;
begin
    Delete(Arr, 0, 1);
end;

procedure TSomeRecUpdate.UpdateArrayWithParam(var ParamArray: TSomeRecArray);
begin
    Delete(ParamArray, 0, 1);
end;

procedure Test;
var r: TSomeRec;
    lArr: TSomeRecArray;
    recUpdate: TSomeRecUpdate;
begin
    lArr := [];

    r.Name := 'abc';
    lArr := lArr + [r];
    r.Name := 'def';
    lArr := lArr + [r];

    recUpdate := TSomeRecUpdate.Create;
    recUpdate.Arr := lArr;
    recUpdate.UpdateArray;
    //(('def'), ('def')) <=== this is the result of copy watch value, WHY two values?

    lArr := [];

    r.Name := 'abc';
    lArr := lArr + [r];
    r.Name := 'def';
    lArr := lArr + [r];

    recUpdate.UpdateArrayWithParam(lArr);

    //(('def')) <=== this is the result of copy watch value - WORKS

    recUpdate.Free;
end;

Answer 1

This is an interesting question!

Since Delete changes the length of the dynamic array -- just as SetLength does -- it has to reallocate the dynamic array. And it also changes the pointer given to it to this new location in memory. But obviously it cannot change any other pointers to the old dynamic array.

So it should decrease the reference count of the old dynamic array and create a new dynamic array with a reference count of 1. The pointer given to Delete will be set to this new dynamic array.

Hence, the old dynamic array should be untouched (except for its reduced reference count, of course). This is essentially documented for the similar SetLength function:

Following a call to SetLength, S is guaranteed to reference a unique string or array -- that is, a string or array with a reference count of one.

But surprisingly, this doesn't quite happen in this case.

Consider this minimal example:

procedure TForm1.FormCreate(Sender: TObject);
var
  a, b: array of Integer;
begin

  a := [$AAAAAAAA, $BBBBBBBB]; {1}
  b := a;                      {2}

  Delete(a, 0, 1);             {3}

end;

I have chosen the values so they are easy to spot in memory (Alt+Ctrl+E).

After (1), a points to $02A2C198 in my test run:

02A2C190  02 00 00 00 02 00 00 00
02A2C198  AA AA AA AA BB BB BB BB

Here the reference count is 2 and the array length is 2, as expected. (See the documentation for the internal data format for dynamic arrays.)

After (2), a = b, that is, Pointer(a) = Pointer(b). They both point to the same dynamic array, which now looks like this:

02A2C190  03 00 00 00 02 00 00 00
02A2C198  AA AA AA AA BB BB BB BB

As expected, the reference count is now 3.

Now, let's see what happens after (3). a now points to a new dynamic array at 2A30F88 in my test run:

02A30F80  01 00 00 00 01 00 00 00
02A30F88  BB BB BB BB 01 00 00 00

As expected, this new dynamic array has a reference count of 1 and only the "B element".

I would expect the old dynamic array, which b is still pointing to, to look as before but with a reduced reference count of 2. But it looks like this now:

02A2C190  02 00 00 00 02 00 00 00
02A2C198  BB BB BB BB BB BB BB BB

Although the reference count is indeed reduced to 2, the first element has been changed.

My conclusion is that

(1) It is part of the contract of the Delete procedure that it invalidates all other references to the initial dynamic array.

or

(2) It should behave as I outlined above, in which case this is a bug.

Unfortunately, the documentation for the Delete procedure doesn't mention this at all.

It feels like a bug.

Update: The RTL Code

I had a look at the source code of the Delete procedure, and this is rather interesting.

It might be helpful to compare the behaviour with that of SetLength (because that one works correctly):

1. If the reference count of the dynamic array is 1, SetLength tries simply to resize the heap object (and update the dynamic array's length field).

2. Otherwise, SetLength makes a new heap allocation for a new dynamic array with a reference count of 1. The reference count of the old array is decreased by 1.

This way, it is guaranteed that the final reference count is always 1 -- either it was that from the beginning or a new array has been created. (It is a good thing that you don't always make a new heap allocation. For instance, if you have a large array with a reference count of 1, simply truncating it is cheaper than copying it to a new location.)

Now, since Delete always makes the array smaller, it is tempting to attempt simply to reduce the size of the heap object where it is. And this is indeed what the RTL code attempts in System._DynArrayDelete. Hence, in your case, the BBBBBBBB is moved to the beginning of the array. All is well.

But then it calls System.DynArraySetLength, which is also used by SetLength. And this procedure contains the following comment,

// If the heap object isn't shared (ref count = 1), just resize it. Otherwise, we make a copy

before it detects that the object is indeed shared (in our case, ref count = 3), makes a new heap allocation for a new dynamic array, and copies the old (reduced) one to this new location. It reduces the ref count of the old array, and updates the ref count, length, and argument pointer of the new one.

So we ended up with a new dynamic array anyway. But the RTL programmers forgot that they had already messed up the original array, which now consists of the new array placed on top of the old one: BBBBBBBB BBBBBBBB.