Haskell, GHC 8: dynamically load/import module

Question

I need to have something like

-- Main.hs
module Main where

main :: IO ()
main = do
  <import Plugin>
  print Plugin.computation

With a Plugin like

-- Plugin.hs
module Plugin where

computation :: Int
computation = 4

However, I need the plugin to be compiled alongside the main application. They need to be deployed together. Only the import (not the compilation) of the module should happen dynamically.

I found Dynamically loading compiled Haskell module - GHC 7.6 along the way and it works just fine with GHC 8.0.2 except for the fact that it requires the source file of the plugin to be in the current working directory when executing the application.

---

Edit (07.12.2017)

Is it possible to load a module from a String instead of a file using the GHC API? http://hackage.haskell.org/package/ghc-8.2.1/docs/GHC.html#t:Target suggests that it's possible, but the documentation has many holes and I can't find a way to actually do this. If this can be accomplished, I can use file-embed to include the plugin source file into the compiled binary. Example:

module Main where

-- Dynamic loading of modules
import GHC
import GHC.Paths ( libdir )
import DynFlags
import Unsafe.Coerce

import Data.Time.Clock (getCurrentTime)
import StringBuffer

pluginModuleNameStr :: String
pluginModuleNameStr = "MyPlugin"

pluginSourceStr :: String
pluginSourceStr = unlines
  [ "module MyPlugin where"
  , "computation :: Int"
  , "computation = 4"
  ]

pluginModuleName :: ModuleName
pluginModuleName = mkModuleName pluginModuleNameStr

pluginSource :: StringBuffer
pluginSource = stringToStringBuffer pluginSourceStr

main :: IO ()
main = do
    currentTime <- getCurrentTime
    defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
      result <- runGhc (Just libdir) $ do
        dflags <- getSessionDynFlags
        setSessionDynFlags dflags
        let target = Target { targetId = TargetModule $ pluginModuleName
                            , targetAllowObjCode = True
                            , targetContents = Just ( pluginSource
                                                    , currentTime
                                                    )
                            }
        setTargets [target]
        r <- load LoadAllTargets
        case r of
          Failed    -> error "Compilation failed"
          Succeeded -> do
            setContext [IIDecl $ simpleImportDecl pluginModuleName]
            result <- compileExpr ("MyPlugin.computation")
            let result' = unsafeCoerce result :: Int
            return result'
      print result

This, however, results in

<command-line>: panic! (the 'impossible' happened)
  (GHC version 8.0.2 for x86_64-apple-darwin):
    module ‘MyPlugin’ is a package module

---

Edit (08.12.2017)

I can compile the "plugin" directly into the final binary by writing the source to a temp file and then loading it like in the linked post (Dynamically loading compiled Haskell module - GHC 7.6). However, this does not play well if the plugin imports packages from Hackage:

module Main where

import Control.Monad.IO.Class (liftIO)
import DynFlags
import GHC
import GHC.Paths (libdir)
import System.Directory (getTemporaryDirectory, removePathForcibly)
import Unsafe.Coerce (unsafeCoerce)

pluginModuleNameStr :: String
pluginModuleNameStr = "MyPlugin"

pluginSourceStr :: String
pluginSourceStr = unlines
  [ "module MyPlugin where"
  , "import Data.Aeson"
  , "computation :: Int"
  , "computation = 4"
  ]

writeTempFile :: IO FilePath
writeTempFile = do
  dir <- getTemporaryDirectory
  let file = dir ++ "/" ++ pluginModuleNameStr ++ ".hs"
  writeFile file pluginSourceStr
  return file

main :: IO ()
main = do
  moduleFile <- writeTempFile
  defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
    result <- runGhc (Just libdir) $ do
      dflags <- getSessionDynFlags
      setSessionDynFlags dflags
      target <- guessTarget moduleFile Nothing
      setTargets [target]
      r <- load LoadAllTargets
      liftIO $ removePathForcibly moduleFile
      case r of
        Failed -> error "Compilation failed"
        Succeeded -> do
          setContext [IIDecl $ simpleImportDecl $ mkModuleName pluginModuleNameStr]
          result <- compileExpr "MyPlugin.computation"
          let result' = unsafeCoerce result :: Int
          return result'
    print result

Is there a way to load packages when, for instance, MyPlugin contains the statement import Data.Aeson? If I add it to the plugin string, it fails with

/var/folders/t2/hp9y8x6s6rs7zg21hdzvhbf40000gn/T/MyPlugin.hs:2:1: error:
    Failed to load interface for ‘Data.Aeson’
    Perhaps you meant Data.Version (from base-4.9.1.0)
    Use -v to see a list of the files searched for.
haskell-loader-exe: panic! (the 'impossible' happened)
  (GHC version 8.0.2 for x86_64-apple-darwin):
  Compilation failed
CallStack (from HasCallStack):
  error, called at app/Main.hs:40:19 in main:Main

---

The reason for my request is database support: We use Persistent to access a database and the dynamic import is needed to support multiple databases (MySQL, PostgreSQL and SQLite) while still allowing the end user to only install one of the three database servers (with other words: not requiring the user to install all of them if they only use, for instance, PostgreSQL). The module that is database-specific should only be loaded when the user actually configures the main application to use that module.

If I don't import Database.Persist.MySQL, then the application does not require MySQL to be installed. Otherwise, the application fails with, for instance,

dyld: Library not loaded: 
/usr/local/opt/mysql/lib/libmysqlclient.20.dylib

on macOS.

Answer 1

A file with a matching module name must exist by the looks of it - it doesn't seem to matter what the file's content is.

On Linux I can even make it be a symlink to /dev/null and things work but a symlink to itself doesn't.