How to store dynamic arrays in a TList?

Question

I need to store an unknown number of groups. Each group has an unknown number of elements/items. This is my 'group':

 TGroup= array of Integer;     <------ dynamic array (as you can see) :)

I want to use a TList to hold my groups. The idea is that I may want to access the groups later and add more items to them.

I have this code, but I can't make it work:

TYPE
   TGroup= array of Integer;                              // Each group has x items (x can be from 1 to 10000)


procedure TForm1.FormCreate(Sender: TObject);
VAR CurGroup: TGroup;
    grp, item: Integer;
    Groups: TList;                                        // can contain up to 1 million groups
begin
 Groups:= TList.Create;

 { Init }
 for grp:= 1 to 4  DO                                     // Put a dummy item in TList
  begin
   SetLength(CurGroup, 1);                                // Create new group
   Groups.Add(@CurGroup);                                 // Store it
  end;

 CurGroup:= NIL;                                          // Prepare for next use

 for grp:= 1 to 4  DO                                     // We create 4 groups. Each group has 3 items
  begin
    CurGroup:= Groups[Groups.Count-1];                    // We retrieve the current group from list in order to add more items to it

    { We add few items }
    for item:= 0 to 2  DO
     begin
       SetLength(CurGroup, Length(CurGroup)+1);           // reserve space for each new item added
       CurGroup[item]:= Item;
     end;

    Groups[Groups.Count-1]:= @CurGroup;                   // We put the group back into the list
  end;

 { Verify }
 CurGroup:= NIL;
 CurGroup:= Groups[0];
 Assert(Length(CurGroup)> 0);                             // FAIL
 if  (CurGroup[0]= 0)
 AND (CurGroup[1]= 1)
 AND (CurGroup[2]= 2)
 then Application.ProcessMessages;                        

 FreeAndNil(Groups);
end;

Note: The code is complete. You can paste it in your Delphi (7) to try it.

Answer 1

Oh, this would be sooooo much nicer in newer versions of Delphi... You'd use the generic, TList<T>. var Groups: TList<TGroup>;

You're best bet is to use another dynamic array: Groups: array of TGroup;

The reason is that dynamic arrays are compiler managed and reference counted. TList only operates on Pointers. There is no straight forward way to keep the reference counts sane when trying to put the dynarrays into the TList.

Another issue you're having is that you're adding the stack address of the dynamic array variable to the TList, and not the actual array. The expression @CurGroup is the "address of the CurGroup variable" which being a local variable, is on the stack.

Answer 2

I've created a wrapper around dynamic array RTTI.

It's just a first draft, but it was inspired by your question, and the fact that the TList methods are missing.

type
  TGroup: array of integer;

var 
  Group: TGroup;
  GroupA: TDynArray;
  i, v: integer;
begin
  GroupA.Init(TypeInfo(TGroup),Group); // associate GroupA with Group
  for i := 0 to 1000 do begin
    v := i+1000; // need argument passed as a const variable
    GroupA.Add(v);
  end;
  v := 1500;
  if GroupA.IndexOf(v)<0 then // search by content
    ShowMessage('Error: 1500 not found!');
  for i := GroupA.Count-1 downto 0 do
    if i and 3=0 then
      GroupA.Delete(i); // delete integer at index i
end;

This TDynArray wrapper will work also with array of string or array of records... Records need only to be packed and have only not reference counted fields (byte, integer, double...) or string reference-counted fields (no Variant nor Interface within).

The IndexOf() method will search by content. That is e.g. for an array of record, all record fields (including strings) must match.

See TDynArray in the SynCommons.pas unit from our Source Code repository. Works from Delphi 6 up to XE, and handle Unicode strings.

And the TTestLowLevelCommon._TDynArray method is the automated unitary tests associated with this wrapper. You'll find out here samples of array of records and more advanced features.

I'm currently implementing SaveToStream and LoadToStream methods...

Perhaps a new way of using generic-like features in all Delphi versions.

Edit:

I've added some new methods to the TDynArray record/object:

• now you can save and load a dynamic array content to or from a string (using LoadFromStream/SaveToStream or LoadFrom/SaveTo methods) - it will use a proprietary but very fast binary stream layout;
• and you can sort the dynamic array content by two means: either in-place (i.e. the array elements content is exchanged) or via an external integer index look-up array (using the CreateOrderedIndex method - in this case, you can have several orders to the same data);
• you can specify any custom comparison function, and there is a new Find method will can use fast binary search if available.

Here is how those new methods work:

var
  Test: RawByteString;
...
  Test := GroupA.SaveTo;
  GroupA.Clear;
  GroupA.LoadFrom(Test);
  GroupA.Compare := SortDynArrayInteger;
  GroupA.Sort;
  for i := 1 to GroupA.Count-1 do
    if Group[i]<Group[i-1] then
      ShowMessage('Error: unsorted!');
  v := 1500;
  if GroupA.Find(v)<0 then // fast binary search
    ShowMessage('Error: 1500 not found!');

Still closer to the generic paradigm, faster, and for Delphi 6 up to XE...