TListView in virtual mode limited to 100,000,000 items?

Question

Setting Items.Count in a ListView to any number over 100,000,000 has the same result as setting the count to 0 - is this a limitation of the underlying windows control, or Delphi specific? I expected the limit to be ~2 billion, since Delphi XE4's documentation says the limit is the size of a (signed) DWORD (ie: 2^31 - 1).

Simple Example:

unit Unit1;

interface

uses
  Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics,
  Vcl.Controls, Vcl.Forms, Vcl.Dialogs, Vcl.ComCtrls;

type
  TForm1 = class(TForm)
    ListView1: TListView;
    procedure ListView1Data(Sender: TObject; Item: TListItem);
    procedure FormCreate(Sender: TObject);
  private
    { Private declarations }
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

procedure TForm1.FormCreate(Sender: TObject);
begin
  // Assumes ListView1.OwnerData := True;
  ListView1.Items.Count := 100000001; // Works if 100000000 is used instead
end;

procedure TForm1.ListView1Data(Sender: TObject; Item: TListItem);
begin
  Item.Caption := Item.Index.ToString();
end;

end.

I did some poking around, sending LVM_SETITEMCOUNT directly to the underlying control, but it returns an error with any LPARAM over 100,000,000 and sets the internal count to 0, leading me to believe it's a limitation of the underlying control. I can't find this documented anywhere - although I assume that having that many items isn't common. Assuming this is a limitation of the control, should probably file a Delphi bug report since no exception gets thrown by TListView when the call fails - it just breaks everything silently.

For now, I'm working around this by keeping the listview out of virtual mode, keeping exactly the number of items that are visible depending on the size of the control (ie: VisibleRowCount property) added to the list, maintaining an offset into my data, and looping over the items in the list to fill the list in essentially the same way as a virtual mode list, using my own scrollbar to control the offset to actually make the limit ~2 billion.

Is there a way around this behavior? Any insight from anyone experienced with working with large amounts of data and a ListView?

Answer 1

Here is my new workaround, it increases the maximum items a virtual ListView can display to 9223372036854775807 (2^63 - 1), in case it is useful to anyone. Think of it as a virtual-virtual ListView. With some work it may be possible to extend it to work with all views, not just list or detailed.

unit Unit1;

interface

uses
  Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics,
  Vcl.Controls, Vcl.Forms, Vcl.Dialogs, Vcl.ComCtrls, Vcl.StdCtrls, System.Math;

type
  TForm1 = class(TForm)
    ListView1: TListView;
    ScrollBar1: TScrollBar;
    procedure ListView1Data(Sender: TObject; Item: TListItem);
    procedure ListView1Resize(Sender: TObject);
    procedure ScrollBar1Change(Sender: TObject);
    procedure FormCreate(Sender: TObject);
    procedure ScrollBar1Scroll(Sender: TObject; ScrollCode: TScrollCode;
      var ScrollPos: Integer);
  private
    { Private declarations }
    Offset: Int64;
    ItemCount: Int64;
    VisibleItems: Integer;
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

// Assumptions:
// ListView1.OwnerData := True
// ListView1.ViewStyle := vsReport with columns set up OR vsList
// ScrollBar1.Min := 0;
// 
// The position of the scrollbar represents equally spaced points along the data
// You can increase this to any number the scrollbar supports
// By default, that means 101 points (0-100), from offset 0 to (ItemCount - VisibleItems + 1)

const
  LISTVIEW_VIRTUALITEMS_MAX = 100000000;

procedure TForm1.FormCreate(Sender: TObject);
begin
  ItemCount := High(Int64); // For testing
  // Make sure the listview shows enough items
  ListView1.Items.Count := Min(ItemCount, LISTVIEW_VIRTUALITEMS_MAX);
end;

procedure TForm1.ListView1Data(Sender: TObject; Item: TListItem);
var
  Index: Int64;
begin
  // Item.Index now represents an offset from an offset, adding them together
  // gives the true index
  Index := Offset + Item.Index;
  Item.Caption := Index.ToString; // Testing
end;

procedure TForm1.ListView1Resize(Sender: TObject);
begin
  VisibleItems := ListView1.VisibleRowCount;
  if VisibleItems = 0 then VisibleItems := 1;
  ListView1.Items.Count := VisibleItems;
end;

procedure TForm1.ScrollBar1Change(Sender: TObject);
begin
  ListView1.Refresh;
end;

procedure TForm1.ScrollBar1Scroll(Sender: TObject; ScrollCode: TScrollCode;
  var ScrollPos: Integer);
  var
  MaxOffset: Int64;
begin
  // Int64 support for scrollbar, etc
  MaxOffset := ItemCount - VisibleItems + 1;
  case ScrollCode of
    TScrollCode.scLineUp: begin
      if Offset > 0 then
        Offset := Offset - 1;
    end;
    TScrollCode.scLineDown: begin
      if Offset < MaxOffset then
        Offset := Offset + 1;
    end;
    scPageUp: begin
      if Offset > VisibleItems then
        Offset := Offset - VisibleItems
      else
        Offset := 0;
    end;
    scPageDown: begin
      if (MaxOffset - Offset) > VisibleItems then
        Offset := Offset + VisibleItems
      else
        Offset := MaxOffset;
    end;
    scPosition, scTrack: begin
      Offset := Trunc((ScrollPos / Scrollbar1.Max) * MaxOffset);
      Exit;
    end;
    scTop: begin
      Offset := 0;
      Exit;
    end;
    scBottom: begin
      Offset := MaxOffset;
      Exit;
    end;
    scEndScroll: begin
    end;
  end;
  ScrollPos := Trunc((Offset / ItemCount) * ScrollBar1.Max);
  ListView1.Refresh;
end;

end.

Answer 2

This is indeed appear a limitation of the underlying control. You will need to either change your UI design to avoid the limit, or find a different control. That said, I doubt that there are many controls that can usefully display 100 million items.