Forwarding data in a DLL

Question

I need to forward a set of symbols from one DLL to another (to support some versioning scheme, PEP 384 if you wonder). It works fine for functions; I write a module definition file, saying

LIBRARY "python3"
EXPORTS
  PyArg_Parse=python32.PyArg_Parse
  PyArg_ParseTuple=python32.PyArg_ParseTuple
  PyArg_ParseTupleAndKeywords=python32.PyArg_ParseTupleAndKeywords
[...]

However, it fails for data. If I say

PyBaseObject_Type=python32.PyBaseObject_Type

then the linker complains that PyBaseObject_Type is an unresolved symbol, even though it actually is exported from python32.dll. Looking at the import library, I notice that, for data, there is only the _imp__ symbol, so I tried

PyBaseObject_Type=python32._imp__PyBaseObject_Type

The linker does actually create a DLL now, however, in this DLL, the forwarding goes to the _imp__ symbol, which then cannot be resolved at runtime. I also tried putting DATA into the line (with or without the _imp__); this doesn't make a difference.

IIUC, forwarding data should work fine, since the data is declared as __declspec(dllimport) for any importer of the DLL, so the compiler should interpret the reference correctly.

So: How can I generate a DLL that does data forwarding?

Answer 1

It seems to me that the solution is not to use DATA during exporting of data from the main DLL (DLL which hold the data).

To reproduce what I mean you can create a project having DllDataForward.c:

#include <Windows.h>

EXTERN_C __declspec(dllexport) int myData = 5;

#ifndef _DEBUG
EXTERN_C BOOL WINAPI _DllMainCRTStartup (HINSTANCE hinstDLL, DWORD fdwReason,
                                         LPVOID lpvReserved)
#else
BOOL WINAPI DllMain (HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
#endif
{
    if (fdwReason == DLL_PROCESS_ATTACH)
        DisableThreadLibraryCalls(hinstDLL);

    return TRUE;
    UNREFERENCED_PARAMETER (lpvReserved);
}

EXTERN_C __declspec(dllexport) BOOL WINAPI MyFunc()
{
    return TRUE;
}

and DllDataForward.def:

LIBRARY "DllDataForward"
EXPORTS
    myData
    MyFunc

Typically one will be use "myData DATA" instead of "myData".

Then you can create a ForwardingDll.c:

#include <Windows.h>

#ifndef _DEBUG
EXTERN_C BOOL WINAPI _DllMainCRTStartup (HINSTANCE hinstDLL, DWORD fdwReason,
                                         LPVOID lpvReserved)
#else
BOOL WINAPI DllMain (HINSTANCE hinstDLL, DWORD fdwReason, LPVOID lpvReserved)
#endif
{
    if (fdwReason == DLL_PROCESS_ATTACH)
        DisableThreadLibraryCalls(hinstDLL);

    return TRUE;
    UNREFERENCED_PARAMETER (lpvReserved);
}

with ForwardingDll.def:

LIBRARY "ForwardingDll" 
EXPORTS
    myNewData=DllDataForward.myData DATA
    MyNewFunc=DllDataForward.MyFunc

You should include import library DllDataForward.lib created during compiling of the DllDataForward as input for the linker during building of the ForwardingDll.dll. Such import library do can be used successfully and you will receive ForwardingDll.dll.

dumpbin.exe ForwardingDll.dll /EXPORTS

produce as the output

...
    ordinal hint RVA      name

          1    0          MyNewFunc (forwarded to DllDataForward.MyFunc)
          2    1          myNewData (forwarded to DllDataForward.myData)
...

A simple test application build using DllDataForward.lib only having the source test.c:

#include <Windows.h>
#include <stdio.h>
#include <tchar.h>

EXTERN_C __declspec(dllimport) int myNewData;
EXTERN_C __declspec(dllimport) BOOL WINAPI MyNewFunc();

int main()
{
    BOOL isSuccess = MyNewFunc();
    int i=myNewData;
    _tprintf (TEXT("i=%d\nisSuccess=%s\n"),
              i, isSuccess? TEXT("TRUE"): TEXT("FALSE"));
}

produce as the output

i=5
isSuccess=TRUE

UPDATED: I want to add a little more information why the usage of "myData DATA" instead of "myData" in the DEF file do a trick and how to use the trick which I suggest with an existing DLL like python32.dll without any changes in python32.dll and without recompiling it. I will show that the original python32.lib miss the exports of all data variables like PyBaseObject_Type. I will show how you can create an additional python32.lib which do has symbols with the data which we need.

First of all I want to clear want changes we have in the import library after the changing from "myData DATA" to "myData" in the DEF file. First let us we compile DllDataForward.dll with the DEF file having "myData DATA" and we look inside of the import library DllDataForward.LIB:

dumpbin.exe DllDataForward.lib /all >%TEMP%\DllDataForward-lib.txt
notepad %TEMP%\DllDataForward-lib.txt

We will see that the lib has 6 public symbols:

  224 __IMPORT_DESCRIPTOR_DllDataForward
  46A __NULL_IMPORT_DESCRIPTOR
  5A8 DllDataForward_NULL_THUNK_DATA
  776 __imp__myData
  708 _MyFunc@0
  708 __imp__MyFunc@0

Next change DEF file from "myData DATA" to "myData", create dll and the import library and look inside it one more time. We will see that now the import library has 7 (!!!) instead of 6 public symbols:

  23A __IMPORT_DESCRIPTOR_DllDataForward
  480 __NULL_IMPORT_DESCRIPTOR
  5BE DllDataForward_NULL_THUNK_DATA
  78C __imp__myData
  78C _myData
  71E _MyFunc@0
  71E __imp__MyFunc@0

So we have problem with the usage of DEF file having "myData DATA" because the created import library not contain the public symbol _myData.

We can stay with the correct DLL having "myData DATA" and create an additional second import library which exports _myData manually. We will make no changes in the DllDataForward.dll, just make and additional lib manually.

To do this we dump the exports of the DllDataForward.dll with respect of dumpbin.exe DllDataForward.dll /exports. We will see:

...
    ordinal hint RVA      name

          1    0 00001020 MyFunc = _MyFunc@0
          2    1 00003000 myData = _myData
...

Now we create the new DllDataForward.def file in another directory based only on the output of dumpbin.exe DllDataForward.dll /exports:

LIBRARY "DllDataForward"
EXPORTS
    myData = _myData

Next using the command

lib.exe /DEF:DllDataForward.def /OUT:DllDataForward.lib /MACHINE:X86

we create the second DllDataForward.lib (in another directory an the original DllDataForward.lib). Now we can compile ForwardingDll.dll using two DllDataForward.lib and receive the DLL which we need. Test.exe will show that the approatch work.

Exactly in the same way we examine python32.lib from the current version 3.2a3:

dumpbin.exe "C:\Program Files\Python32\libs\python32.lib" /all >python32-lib.txt
notepad python32-lib.txt

we will find out following lines (about at the beginning of the file)

1957 public symbols
…
1BCCC _PyArg_Parse
1BCCC __imp__PyArg_Parse
…
1BFF6 __imp__PyBaseObject_Type
…

We can also verify with

dumpbin C:\Windows\system32\python32.dll /exports >%TEMP%\python32-exports.txt
notepad %TEMP%\python32-exports.txt

that the symbol PyBaseObject_Type will be exported as

 14    D 001DD5D0 PyBaseObject_Type

So we can create additional python32.lib from the python32.def file

LIBRARY "python32"
EXPORTS
    PyBaseObject_Type

using

lib /DEF:python32.def /OUT:python32.lib /MACHINE:X86

Now you can define the DEF of your dll

LIBRARY "python3"
EXPORTS
  PyArg_Parse=python32.PyArg_Parse
  PyArg_ParseTuple=python32.PyArg_ParseTuple
  PyArg_ParseTupleAndKeywords=python32.PyArg_ParseTupleAndKeywords
  PyBaseObject_Type=python32.PyBaseObject_Type DATA

exactly like you want originally, but we will use both "C:\Program Files\Python32\libs\python32.lib" and the small second python32.lib which we created during linking.

I don't use python myself and don't know the size of PyBaseObject_Type, but if I declare it as int

EXTERN_C __declspec(dllimport) int PyBaseObject_Type;

I can verify that the first part of PyBaseObject_Type is 1. It work!

Sorry for the long answer and thanks for all who read all my answer till this place.

Answer 2

I played around with this a bit and was not able to find the correct combination that worked with a .def file. However, I was able to forward both functions and data via the following pragmas in the source of the forwarding DLL:

#pragma comment(linker,"/export:_data=org.data,DATA")
#pragma comment(linker,"/export:_func=org.func")

Note I had to use the decorated data and function names.

Below are the files for a complete example:

org.c

int data = 5;
int func(int a)
{
    return a * 2;
}

org.def

EXPORTS
    data DATA
    func

fwd.c

#pragma comment(linker,"/export:_data=org.data,DATA")
#pragma comment(linker,"/export:_func=org.func")

int func2(int a)
{
    return a + 2;
}

fwd.def

EXPORTS
    func2

example.c

#include <stdio.h>

__declspec(dllimport) int data;
__declspec(dllimport) int func(int a);
__declspec(dllimport) int func2(int a);

int main()
{
    printf("data=%d func(5)=%d func2(5)=%d\n",data,func(5),func2(5));
    return 0;
}

makefile

all: example.exe org.dll

example.exe: example.c fwd.dll
    cl /W4 example.c /link fwd.lib

org.dll: org.c
    cl /LD /W4 org.c org.def

fwd.dll: fwd.c
    cl /LD /W4 fwd.c fwd.def

clean:
    del *.exe *.dll *.obj *.exp *.lib