How to work around duplicate symbol error when using Yesod and Darcs library?

Question

It seems impossible to use Yesod together with the Darcs library due to a linker problem. I tracked down the problem and need hints to work around it by people familiar with Darcs internals.

When using the darcs library in a Yesod application, I get the following error:

GHCi runtime linker: fatal error: I found a duplicate definition for symbol
   sha256_init
whilst processing object file
   /home/sebfisch/.cabal/lib/darcs-2.9.5/ghc-7.4.2/libHSdarcs-2.9.5.a
This could be caused by:
   * Loading two different object files which export the same symbol
   * Specifying the same object file twice on the GHCi command line
   * An incorrect `package.conf' entry, causing some object to be
     loaded twice.
GHCi cannot safely continue in this situation.  Exiting now.  Sorry.

It appears to be caused by the darcs and cryptohash libraries exposing the same symbol, as searching through the respective object files reveals:

# for file in `find ~/.cabal/lib/ -name "*.a"`; do (readelf -s $file | grep -i sha256_init) && (echo $file; echo); done
   293: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND sha256_init
    17: 0000000000000690    94 FUNC    GLOBAL DEFAULT    1 sha256_init
~/.cabal/lib/cryptohash-0.7.5/ghc-7.4.2/libHScryptohash-0.7.5.a

    10: 0000000000000290    45 FUNC    GLOBAL DEFAULT    1 sha256_init
~/.cabal/lib/darcs-2.8.2/ghc-7.4.2/libHSdarcs-2.8.2.a

I wrote a test program to confirm that the darcs and cryptohash libraries are in conflict:

import           Crypt.SHA256          (sha256sum)
import           Crypto.Hash.SHA256    (hash)
import           Data.ByteString       (empty)
import qualified Data.ByteString.Char8 as BS

main :: IO ()
main = do
    BS.putStrLn $ hash empty    -- cryptohash
    putStrLn $ sha256sum empty  -- darcs

It fails to compile with a similar error:

/home/sebfisch/.cabal/lib/cryptohash-0.7.5/ghc-7.4.2/libHScryptohash-0.7.5.a(sha256.o): In function `sha256_update': sha256.c:(.text+0x4b0): multiple definition of `sha256_update'
/home/sebfisch/.cabal/lib/darcs-2.8.2/ghc-7.4.2/libHSdarcs-2.8.2.a(sha2.o):sha2.c:(.text+0xf90): first defined here
/home/sebfisch/.cabal/lib/cryptohash-0.7.5/ghc-7.4.2/libHScryptohash-0.7.5.a(sha256.o): In function `sha224_update': sha256.c:(.text+0x640): multiple definition of `sha224_update'
/home/sebfisch/.cabal/lib/darcs-2.8.2/ghc-7.4.2/libHSdarcs-2.8.2.a(sha2.o):sha2.c:(.text+0xbb0): first defined here
/home/sebfisch/.cabal/lib/cryptohash-0.7.5/ghc-7.4.2/libHScryptohash-0.7.5.a(sha256.o): In function `sha256_init': sha256.c:(.text+0x690): multiple definition of `sha256_init'
/home/sebfisch/.cabal/lib/darcs-2.8.2/ghc-7.4.2/libHSdarcs-2.8.2.a(sha2.o):sha2.c (.text+0x290): first defined here
/home/sebfisch/.cabal/lib/cryptohash-0.7.5/ghc-7.4.2/libHScryptohash-0.7.5.a(sha256.o): In function `sha224_init': sha256.c:(.text+0x6f0): multiple definition of `sha224_init'
/home/sebfisch/.cabal/lib/darcs-2.8.2/ghc-7.4.2/libHSdarcs-2.8.2.a(sha2.o):sha2.c (.text+0x620): first defined here
collect2: ld returned 1 exit status

The cryptohash library is required by yesod-static and cannot easily be avoided when writing a Yesod application. How can I use Yesod and Darcs (as a library) in the same application?

Would it help to delete the duplicate symbols from one library? Both packages access hashing functions via FFI but using different files.

From darcs/Crypt.SHA256:

foreign import ccall unsafe "sha2.h sha256" c_sha256
    :: Ptr CChar -> CSize -> Ptr Word8 -> IO ()

From cryptohash/Crypto.Hash.SHA256:

foreign import ccall unsafe "sha256.h sha256_init"
    c_sha256_init :: Ptr Ctx -> IO ()

foreign import ccall "sha256.h sha256_update"
    c_sha256_update :: Ptr Ctx -> CString -> Word32 -> IO ()

foreign import ccall unsafe "sha256.h sha256_finalize"
    c_sha256_finalize :: Ptr Ctx -> CString -> IO ()

Another idea is to rewrite Darcs to not use its own hashing function. How can I reimplement Darcs's SHA256 module to use cryptohash? The two statements in the main function of my test program do not give the same output (tested by commenting out the other statement), so using cryptohash in Darcs does not seem entirely straightforward.

Answer 1

The darcs hash output is just the base16 encoded version of the cryptohash output. Looks like base16-bytestring is one way to bridge that gap. I tried it and Crypt.SHA256 becomes as simple as:

module Crypt.SHA256 ( sha256sum ) where

import Crypto.Hash.SHA256 ( hash )
import Data.ByteString ( ByteString )
import Data.ByteString.Base16 ( encode )
import Data.ByteString.Char8 ( unpack )

sha256sum :: ByteString -> String
sha256sum = unpack . encode . hash

In fact the hashed-storage package also has a copy of sha2.c and fixed the problem by renaming the symbols. So the simplest quick fix for darcs 2.8 is to copy sha2.h and sha2.c from hashed-storage, replace hashed_storage_ with darcs_ in both files, and change the FFI import in src/Crypt/SHA256.hs in darcs to:

foreign import ccall unsafe "sha2.h darcs_sha256" c_sha256
    :: Ptr CChar -> CSize -> Ptr Word8 -> IO ()

I'd be happy to release darcs 2.8.3 with this change if it would help you. For 2.10 I'll switch to using cryptohash as above as I don't see any reason to keep using the local C version and in general in darcs we are trying to get rid of private implementations of common code.

EDIT: I originally thought hashed-storage would have the same problem, but I was wrong (in retrospect it's obvious that it would have clashed with darcs itself if not for the renaming).