Statically linking a C library with a Haskell library

Question

I have a Haskell project that aims to create some C++ bindings. I've written the C wrappers and compiled them into a stand-alone statically linked library.

I'd like to write the Haskell bindings to link statically to the C wrappers so that I don't have to distribute the C wrappers separately but I can't seem to get it working and would appreciate some help.

I specify the C library as an extra library but my cabal build step doesn't seem to add it to compile command.

I've created a small project to illustrate this (http://github.com/deech/CPlusPlusBindings).

It contains a small C++ class (https://github.com/deech/CPlusPlusBindings/tree/master/cpp-src), the C wrapper (https://github.com/deech/CPlusPlusBindings/tree/master/c-src), a working C test routine (https://github.com/deech/CPlusPlusBindings/tree/master/c-test) and the Haskell file (https://github.com/deech/CPlusPlusBindings/blob/master/src/BindingTest.chs).

The C library is added in Setup.hs not in the Cabal file because that's how I have it my real project which builds the C library using "make" through Cabal just before the build stepf. I have verified that at the build step the extraLibs part of BuildInfo contains the library name and extraLibDirs contains the right directory.

The output of my cabal build is:

creating dist/setup
./dist/setup/setup build --verbose=2
creating dist/build
creating dist/build/autogen
Building CPlusPlusBinding-0.1.0.0...
Preprocessing library CPlusPlusBinding-0.1.0.0...
Building library...
creating dist/build
/usr/local/bin/ghc --make -fbuilding-cabal-package -O -odir dist/build -hidir dist/build -stubdir dist/build -i -idist/build -isrc -idist/build/autogen -Idist/build/autogen -Idist/build -I/home/deech/Old/Haskell/CPlusPlusBinding/c-src -I/home/deech/Old/Haskell/CPlusPlusBinding/cpp-includes -optP-include -optPdist/build/autogen/cabal_macros.h -package-name CPlusPlusBinding-0.1.0.0 -hide-all-packages -package-db dist/package.conf.inplace -package-id base-4.6.0.1-8aa5d403c45ea59dcd2c39f123e27d57 -XHaskell98 -XForeignFunctionInterface BindingTest
Linking...
/usr/bin/ar -r dist/build/libHSCPlusPlusBinding-0.1.0.0.a dist/build/BindingTest.o
/usr/bin/ar: creating dist/build/libHSCPlusPlusBinding-0.1.0.0.a
/usr/bin/ld -x --hash-size=31 --reduce-memory-overheads -r -o dist/build/HSCPlusPlusBinding-0.1.0.0.o dist/build/BindingTest.o
In-place registering CPlusPlusBinding-0.1.0.0...
/usr/local/bin/ghc-pkg update - --global --user --package-db=dist/package.conf.inplace

Unfortunately neither the compilation nor the linking step uses the C library. There are no other warnings or errors.

Answer 1

To solve this problem I had to:

1. re-link the Haskell library with the C bindings' object files and
2. use the ghc-options tag in my Cabal file to make sure they linked in the right order.

All the changes are in the test project (http://github.com/deech/CPlusPlusBindings).

Below the process of creating a new archive that includes both the C and Haskell objects is explained in detail and it is not simple. The complexity occurs because there is no way (as of Cabal 1.16.0.2) to hook into the linker part of the build process.

Setting the flags in the Cabal file is trivial so it is not described here.

Relinking The Haskell Library

1. Set the build type to custom by adding:

   build-type: custom
   

   to the cabal file.

2. Insert customized build logic by replacing the main method in Setup.hs with:

   main = defaultMainWithHooks simpleUserHooks {
                 buildHook = myBuildHook, 
                 ...
          }
   

   This tells the build process that instead of going with the default build process defined in simpleUserHooks it should use the myBuildHook function which is defined below. Similarly the clean up process is overridden with the custom function myCleanHook.

3. Define the build hook. This build hook will run make on the command line to build the C++, and C portions and then use the C object files when creating linking the Haskell bindings.

   We start off myBuildHook:

   myBuildHook pkg_descr local_bld_info user_hooks bld_flags = do
   

   by first running make with no arguments:

   rawSystemExit normal "make" []
   

   Then add the locations of the header files and library directories and the library itself to the PackageDescription record and update the LocalBuildInfo with the new package description:

   let new_pkg_descr = (addLib . addLibDirs . addIncludeDirs $ pkg_descr)
       new_local_bld_info = local_bld_info {localPkgDescr = new_pkg_descr}
   

   Before the buildHook fired the configureHook stored the order of compilation in the compBuildOrder (component build order) key of the LocalBuildInfo record. We need to isolate the building of the library so we separate the library building and executable building parts of the build process.

   The build order is just a list and we know the build component is a library if it's just a plain CLibName type constructor so we isolate those elements from the list and update the LocalBuildInfo record with only them:

   let (libs, nonlibs) = partition
                          (\c -> case c of
                                   CLibName -> True
                                   _ -> False)
                          (compBuildOrder new_local_bld_info)
       lib_lbi = new_local_bld_info {compBuildOrder = libs}
   

4. Now we run the default build hook with the updated records:

   buildHook simpleUserHooks new_pkg_descr lib_lbi user_hooks bld_flags
   

5. Once it's done building an archive has been created but we have to re-create it to include the C objects generated by the make command in step 1. So we grab some settings and a list of the C object file paths:

   let verbosity = fromFlag (buildVerbosity bld_flags)
   info verbosity "Relinking archive ..."
   let pref = buildDir local_bld_info
       verbosity = fromFlag (buildVerbosity bld_flags)
   cobjs <- getLibDirContents >>= return . map (\f -> combine clibdir f) 
                                         . filter (\f -> takeExtension f == ".o")
   

   And then hand it off to withComponentsLBI which acts on each component of the build. In this case since we're only dealing with the library part there is only one component. Cabal provides getHaskellObjects for getting a list of the Haskell object files and createArLibArchive for creating an archive so we can re-run the linker:

    withComponentsLBI pkg_descr local_bld_info $ \comp clbi ->
     case comp of
       (CLib lib) -> do
                 hobjs <- getHaskellObjects lib local_bld_info pref objExtension True
                 let staticObjectFiles = hobjs ++ cobjs
                 (arProg, _) <- requireProgram verbosity arProgram (withPrograms local_bld_info)
                 let pkgid = packageId pkg_descr
                     vanillaLibFilePath = pref </> mkLibName pkgid
                 Ar.createArLibArchive verbosity arProg vanillaLibFilePath staticObjectFiles
       _ -> return ()
   

6. The default buildHook which was run in Step 4 created a temporary package database file named "package.conf.inplace" which holds the description of the library that was built so that executable can link against it without the library needing to be installed to the default system package file. Unfortunately every buildHook run blanks it out so we need to hold on to a temporary copy:

     let distPref  = fromFlag (buildDistPref bld_flags)
         dbFile = distPref </> "package.conf.inplace"
    (tempFilePath, tempFileHandle) <- openTempFile distPref "package.conf"
    hClose tempFileHandle
    copyFile dbFile tempFilePath
   

7. Now we store a path to that copy into the LocalBuildInfo structure along with the executable parts of the build process which were filtered out in Step 3.

     let exe_lbi = new_local_bld_info {
                     withPackageDB = withPackageDB 
                                       new_local_bld_info ++ 
                                      [SpecificPackageDB tempFilePath], 
                     compBuildOrder = nonlibs
                   }
   

   and store the path again in the extraTmpFiles part of the PackageDescription so it can be removed by the default clean up hook.

   exe_pkg_descr = new_pkg_descr {extraTmpFiles = extraTmpFiles new_pkg_descr ++ [tempFilePath]}
   

8. Now we finally run the default buildHook again with the updated records (which now know about the new archive) on just the executable components:

   buildHook simpleUserHooks exe_pkg_descr exe_lbi user_hooks bld_flags