Why does TEnumerable<T> use pass-through methods?

Question

TEnumerable<T>, the base class for all the Generics.Collections container classes, has a very strange declaration. It looks like this:

type
  TEnumerable<T> = class abstract
  protected
    function DoGetEnumerator: TEnumerator<T>; virtual; abstract;
  public
    function GetEnumerator: TEnumerator<T>;
  end;

function TEnumerable<T>.GetEnumerator: TEnumerator<T>;
begin
  Result := DoGetEnumerator;
end;

TEnumerator<T> likewise declares a public MoveNext method and a private DoMoveNext function, and MoveNext does nothing but call DoMoveNext.

Can anyone explain to me just what purpose this serves, aside from adding extra function call overhead, making call stacks longer, and creating confusion in the minds of coders attempting to inherit from these base classes? Is there any actual advantage to this way of structuring it, because if there is I don't see it...

Answer 1

Disclaimer: I wrote TEnumerable<T>. If I were to do it again, I'd probably have written it with less performance and more simplicity in mind, as I've learned this optimization confuses a lot of people.

It's designed to avoid a virtual call in for-in loops while maintaining compatibility with polymorphism. This is the general pattern:

• Base class Base defines protected virtual abstract method V and public non-virtual method M. M dispatches to V, so polymorphic calls via a Base-typed variable will route to overridden behaviour of V.

• Descendant classes such as Desc implement static override of M (hiding Base.M) which contains the implementation, and implement an override of V which calls Desc.M. Calls of M via Desc-typed variables go direct to the implementation without a virtual dispatch.

Concrete example: when the compiler generates code for this sequence:

var
  someCollection: TSomeCollection<TFoo>;
  x: TFoo;
begin
  // ...
  for x in someCollection do
    // ...
end;

... the compiler looks for a method called GetEnumerator on the static type of someCollection, and a method called MoveNext on the type it returns (and similarly for Current property). If this method has static dispatch, a virtual call can be eliminated.

This is most important for loops, and thus MoveNext / Current accessor. But in order for the optimization to work, the return type of the GetEnumerator method must be covariant, that is, it needs to statically return the correct derived enumerator type. But in Delphi, unlike C++ [1], it is not possible to override an ancestor method with a more-derived return-type, so the same trick needs to be applied for a different reason, to change the return type in descendants.

The optimization also potentially permits inlining of the MoveNext and GetCurrent method calls, as it is very difficult for a static compiler to "see through" virtual calls and still be fast.

[1] C++ support return-value covariance on overridden methods.

Answer 2

The GetEnumerator() method isn't virtual; you can't override it. It's one way of ensuring that GetEnumerator() will always exist, always take a fixed set of parameters (none in this case) and that some programmer won't mess it up for descendant classes. Anyone that uses TEnumerable - or a descendant - can call GetEnumerator().

But since there will be different TEnumerable descendants that do different things, the DoGetEnumerator() allows a programmer to make changes internal to the structure. The "virtual" allows the method to be overridden. The "abstract" forces descendant classes to implement the method - the compiler won't allow you to forget. And since DoGetEnumerator() is declared as protected (at least at this level), a programmer using a TEnumerable descendant can't bypass GetEnumerator() and call DoGetEnumerator() directly.