Using GHC API to compile Haskell sources to CORE and CORE to binary

Question

The Idea

Hello! I want to create a program, that will generate Haskell Core and will use GHC API to compile it further into an executable. But before I will do it I want to construct a very basic example, showing how can we just compile Haskell sources into CORE and then into the binary file.

The problem

I have read a lot of documentation and tried many methods from GHC Api, but for now without success. I started with Official GHC Api introduction and successfully compiled the examples. The examples show the usage of the following functions: parseModule, typecheckModule, desugarModule, getNamesInScope and getModuleGraph but does not cover the final compilation step. On the other hand, there are some functions in the api, whose names look related to the problem, like HscMain.{hscCompileOneShot, hscCompileBatch} or GHC.{compileToCoreModule, compileCoreToObj}. I tried to use them, but I get runtime errors, like in this example:

import GHC
import GHC.Paths ( libdir )
import DynFlags
targetFile = "Test.hs"

main :: IO ()
main = do
   res <- example
   return ()

example = 
    defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
      runGhc (Just libdir) $ do
        dflags <- getSessionDynFlags
        let dflags' = foldl xopt_set dflags
                            [Opt_Cpp, Opt_ImplicitPrelude, Opt_MagicHash]
        setSessionDynFlags dflags'
        coreMod <- compileToCoreModule targetFile
        compileCoreToObj False coreMod "foo" "bar"
        return () 

which can be compiled with ghc -package ghc Main.hs and which results in the following error during runtime:

Main: panic! (the 'impossible' happened)
  (GHC version 7.8.3 for x86_64-unknown-linux):
    expectJust mkStubPaths

which of course can be the result of wrong API usage, in particular, because of line compileCoreToObj False coreMod "foo" "bar", wher the string are just random ones, because the documentation does not say much about them. If we look into the sources, it seems, that the first one is the output name and the second one is "extCore_filename", whatever it could be.

Another worrying thing is the comment in the documentation next to the compileCoreToObj function:

[...] This has only so far been tested with a single self-contained module.

But I hope it will not introduce any further problems.

The question

What is the best possible way to create this solution? How can we create a minimal working example, that will load haskell sources, compile them into the CORE and then compile the core to final executable (using the GHC API). The intermediate step is needed for further replacement by custom CORE.

As a side-question - is it currently possible to provide GHC with external core files or this feature is not implemented yet and I will have to construct the Core manually, using GHC.Api (related to: Compiling to GHC Core)

Update

I was finally able to create a small example allowing loading a module and compiling it to .hi and .o files. This is not a solution for the problem, because it does not allow me to replace the CORE and it does not link the object files into executables yet:

import GHC
import GHC.Paths ( libdir )
import DynFlags
import Linker
import Module
targetFile = "Test.hs"

main :: IO ()
main = do
   res <- example
   return ()

example = 
    defaultErrorHandler defaultFatalMessager defaultFlushOut $ do
      runGhc (Just libdir) $ do
        dflags <- getSessionDynFlags
        let dflags2 = dflags { ghcLink   = LinkBinary
                             , hscTarget = HscAsm
                             }
        let dflags' = foldl xopt_set dflags2
                            [Opt_Cpp, Opt_ImplicitPrelude, Opt_MagicHash]
        setSessionDynFlags dflags'
        setTargets =<< sequence [guessTarget "Test.hs" Nothing]

        load LoadAllTargets
        return ()

Answer 1

Generating textual representation of core is not a problem here, because it can be made in multiple ways. You can use -fext-core flag to generate .hcr files and work with them using e.g. extcore. There are also other packages that can dump core, like ghc-core or ghc-core-html.

The main problem here, is loading ghc-core into ghc. As far as I know it was supported but now is not, because there was low interest in using it and it became obsolete with time.

The best thing we can try here is dig more into ghc internals, find places where ghc-core is used and try to modify it there. Maybe we can also try creating a ghc plugin and modify core with it.