Callback from .Net COM dll to Delphi client in registration-free (side-by-side) COM

Question

TLDR: I'm trying to call async callbacks from a .Net COM dll to Delphi client .exe, but these does not seem to work properly in registration-free COM, while synchronous callbacks do work, and also async callbacks are working when it's not a reg-free COM.

---

My global case is that I'm having a foreign closed-source .Net dll that exposes some public events. I need to pass these events to Delphi app. So I decided to make an intermediate .dll that would work as a COM bridge between my app and that another dll. It worked just fine when my dll is registered via regasm, but things are getting worse when I switch to reg-free COM. I shortened my case to small reproducible example which does not depend on the other dll, so I'll be posting it below.

Based on this answer I made a public interface ICallbackHandler which I expect to get from Delphi client app:

namespace ComDllNet
{
    [ComVisible(true)]
    [Guid("B6597243-2CC4-475B-BF78-427BEFE77346")]
    [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
    public interface ICallbackHandler
    {
        void Callback(int value);
    }

    [ComVisible(true)]
    [Guid("E218BA19-C11A-4303-9788-5A124EAAB750")]
    public interface IComServer
    {
        void SetHandler(ICallbackHandler handler);
        void SyncCall();
        void AsyncCall();
    }

    [ComVisible(true)]
    [Guid("F25C66E7-E9EF-4214-90A6-3653304606D2")]
    [ClassInterface(ClassInterfaceType.None)]
    public sealed class ComServer : IComServer
    {
        private ICallbackHandler handler;
        public void SetHandler(ICallbackHandler handler) { this.handler = handler; }

        private int GetThreadInfo()
        {
            return Thread.CurrentThread.ManagedThreadId;
        }

        public void SyncCall()
        {
            this.handler.Callback(GetThreadInfo());
        }

        public void AsyncCall()
        {
            this.handler.Callback(GetThreadInfo());
            Task.Run(() => {
                for (int i = 0; i < 5; ++i)
                {
                    Thread.Sleep(500);
                    this.handler.Callback(GetThreadInfo());
                }
            });
        }
    }
}

Then, I gave a strong name to dll, and registered it via Regasm.exe.

Now I turned to Delphi client. I create the tlb wrapper code using Component > Import Component > Import a Type Library which gave me

  ICallbackHandler = interface(IUnknown)
    ['{B6597243-2CC4-475B-BF78-427BEFE77346}']
    function Callback(value: Integer): HResult; stdcall;
  end;
  IComServer = interface(IDispatch)
    ['{E218BA19-C11A-4303-9788-5A124EAAB750}']
    procedure SetHandler(const handler: ICallbackHandler); safecall;
    procedure SyncCall; safecall;
    procedure AsyncCall; safecall;
  end;
  IComServerDisp = dispinterface
    ['{E218BA19-C11A-4303-9788-5A124EAAB750}']
    procedure SetHandler(const handler: ICallbackHandler); dispid 1610743808;
    procedure SyncCall; dispid 1610743809;
    procedure AsyncCall; dispid 1610743810;
  end;

And created a handler and some Form with two buttons and memo to test things:

unit Unit1;

interface

uses
  Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
  Dialogs, ComDllNet_TLB, StdCtrls;

type
  THandler = class(TObject, IUnknown, ICallbackHandler)
  private
    FRefCount: Integer;
  protected
   function Callback(value: Integer): HResult; stdcall;

   function QueryInterface(const IID: TGUID; out Obj): HRESULT; stdcall;
   function _AddRef: Integer; stdcall;
   function _Release: Integer; stdcall;
  public
    property RefCount: Integer read FRefCount;
  end;

type
  TForm1 = class(TForm)
    Memo1: TMemo;
    syncButton: TButton;
    asyncButton: TButton;
    procedure FormCreate(Sender: TObject);
    procedure syncButtonClick(Sender: TObject);
    procedure asyncButtonClick(Sender: TObject);
  private
    { Private declarations }
    handler : THandler;
    server : IComServer;
  public
    { Public declarations }
  end;

var
  Form1: TForm1;

implementation

{$R *.dfm}

function THandler._AddRef: Integer;
begin
  Inc(FRefCount);
  Result := FRefCount;
end;

function THandler._Release: Integer;
begin
  Dec(FRefCount);
  if FRefCount = 0 then
  begin
    Destroy;
    Result := 0;
    Exit;
  end;
  Result := FRefCount;
end;

function THandler.QueryInterface(const IID: TGUID; out Obj): HRESULT;
const
  E_NOINTERFACE = HRESULT($80004002);
begin
  if GetInterface(IID, Obj) then
    Result := 0
  else
    Result := E_NOINTERFACE;
end;

function THandler.Callback(value: Integer): HRESULT;
 begin
  Form1.Memo1.Lines.Add(IntToStr(value));
  Result := 0;
 end;

procedure TForm1.FormCreate(Sender: TObject);
 begin
  handler := THandler.Create();
  server := CoComServer.Create();
  server.SetHandler(handler);
 end;

procedure TForm1.syncButtonClick(Sender: TObject);
 begin
  Form1.Memo1.Lines.Add('Begin sync call');
  server.SyncCall();
  Form1.Memo1.Lines.Add('End sync call');
 end;

procedure TForm1.asyncButtonClick(Sender: TObject);
 begin
  Form1.Memo1.Lines.Add('Begin async call');
  server.AsyncCall();
  Form1.Memo1.Lines.Add('End async call');
 end;

end.

So, I run it, pressed 'sync' and 'async' buttons and everything worked as expected. Note how the thread ids of a Task comes after 'End async call' line (also with some delay because of Thread.Sleep):

End of part one. Now I switched to using Rregistration-free (side-by-side) COM. Based on this answer I added dependentAssembly part to my Delphi app manifest:

<dependency>
    <dependentAssembly>
        <assemblyIdentity name="ComDllNet" version="1.0.0.0" publicKeyToken="f31be709fd58b5ba" processorArchitecture="x86"/>
    </dependentAssembly>
</dependency>

Using the mt.exe tool I generated a manifest for my dll:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<assembly xmlns="urn:schemas-microsoft-com:asm.v1" manifestVersion="1.0">
    <assemblyIdentity name="ComDllNet" version="1.0.0.0" publicKeyToken="f31be709fd58b5ba" processorArchitecture="x86"/>
    <clrClass clsid="{F25C66E7-E9EF-4214-90A6-3653304606D2}" progid="ComDllNet.ComServer" threadingModel="Both" name="ComDllNet.ComServer" runtimeVersion="v4.0.30319"/>
    <file name="ComDllNet.dll" hashalg="SHA1"/>
</assembly>

Then I unregistered the dll and run the app. And I found that only synchronous parts of the callbacks are working:

Edit: Note that you have to unregister with /tlb option, otherwise it will continue working on local machine, as if dll was still registered (see).

I tired a number of things already, and I'm not sure what to do next. I'm staring to suspect that the initial approach should not work at all and I need to implement some threading on the Delphi app side. But I'm not sure what and how. Any help would be appreciated!

Answer 1

You have to register the ICallbackHandler interface. So, in the same file where you have the clrClass element, but as a sibling of the file elements, add:

    <comInterfaceExternalProxyStub iid="{B6597243-2CC4-475B-BF78-427BEFE77346}"
                                   name="ICallbackHandler"
                                   tlbid="{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}"
                                   proxyStubClsid32="{00020424-0000-0000-C000-000000000046}"/>

This tells COM to use an external proxy/stub, the type library marshaler ({00020424-0000-0000-C000-000000000046}), and it tells the type library marshaler to look for your type library ({XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}). This GUID is your assembly's GUID, found in your project's properties (check AssemblyInfo.cs).

You need to generate this type library. Since you want registration-free COM, I think TLBEXP.EXE fits the bill perfectly, you can set it up as a post build event.

Finally, you can keep a separate type library file or you can embed it in your assembly. I advise you keep it separate, even more so if your assembly is big.

Either way, you need to put this into the manifest. Here's an example using a separate .TLB file:

    <file name="ComDllNet.tlb">
        <typelib tlbid="{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}"
                 version="1.0"
                 helpdir="."
                 flags=""/>
    </file>

If you embed the type library, add the following as a child of the <file name="ComDLLNet.dll"/> element:

        <typelib tlbid="{XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX}"
                 version="1.0"
                 helpdir="."
                 flags=""/>

Answer 2

This is too long to be a comment, so posting it as an answer.

A pointer to COM interfaces should never be accessed from a different COM apartment without proper marshaling. In this case, this.handler is (most likely) an STA COM object created on an Delphi's STA thread. Then it directly gets called from a .NET MTA pool thread thread inside Task.Run, without any kind of COM marshaling. This is a violation of COM hard rules, outlined here INFO: Descriptions and Workings of OLE Threading Models.

The same is true about a managed RCW proxy wrapping a COM interface on the .NET side. The RCW will just marshal the method call from managed to unmanaged code, but it won't do anything about COM marshaling.

This can lead to all kind of nasty surprised, especially if the OP accesses the Delphi app's UI inside handler.Callback.

Now, it's possible that the handler object aggregates the Free Threaded Marshaler (this would have its own rules to follow, and I doubt it's the case with the OP code). Let it be so, the pointer to handler object will indeed be unmarshaled to the same pointer by the FTM. However, the server code which calls the object from another thread (i.e., Task.Run(() => { ... this.handler.Callback(GetThreadInfo() ...}) should never assume the COM object is free-threaded, and it still should do the correct marshaling. If lucky, the direct pointer will be given back when unmarshaling.

There's a bunch of methods to do the marshaling:

• CoMarshalInterThreadInterfaceInStream/CoGetInterfaceAndReleaseStream.
• CoMarshalInterface/CoUnmarshalInterface.
• Global Interface Table (GIT).
• CreateObjrefMoniker/BindMoniker.
• etc.

Of course, for the above marshaling methods to work, the correct COM proxy/stub classes should be registered or provisioned via a side-by-side manifest, as Paulo Madeira's answer explains.

Alternatively, a custom dispinterface can be used (in which case all calls would go through IDispatch with OLE Automation marshaler), or any other standard COM interface known to the standard COM marshaler. I often use IOleCommandTarget for simple callbacks, it doesn't require anything to be registered.