Debugging HXT performance problems

Question

I'm trying to use HXT to read in some big XML data files (hundreds of MB.)

My code has a space-leak somewhere, but I can't seem to find it. I do have a little bit of a clue as to what is happening thanks to my very limited knowledge of the ghc profiling tool chain.

Basically, the document is parsed, but not evaluated.

Here's some code:

{-# LANGUAGE Arrows, NoMonomorphismRestriction #-}

import Text.XML.HXT.Core
import System.Environment (getArgs)
import Control.Monad (liftM)

main = do file <- (liftM head getArgs) >>= parseTuba
          case file of(Left  m) -> print "Failed."
                      (Right _) -> print "Success."

data Sentence t = Sentence [Node t] deriving Show
data Node t = Word { wSurface :: !t } deriving Show

parseTuba :: FilePath -> IO (Either String ([Sentence String]))
parseTuba f = do r <- runX (readDocument [] f >>> process)
                 case r of
                      []   -> return $ Left "No parse result."
                      [pr] -> return $ Right pr
                      _    -> return $ Left "Ambiguous parse result!"

process :: (ArrowXml a) => a XmlTree ([Sentence String])
process = getChildren >>> listA (tag "sentence" >>> listA word >>> arr (\ns -> Sentence ns))

word :: (ArrowXml a) => a XmlTree (Node String)
word = tag "word" >>> getAttrValue "form" >>> arr (\s -> Word s)

-- | Gets the tag with the given name below the node.
tag  :: (ArrowXml a) => String -> a XmlTree XmlTree
tag s = getChildren >>> isElem >>> hasName s

I'm trying to read a corpus file, and the structure is obviously something like <corpus><sentence><word form="Hello"/><word form="world"/></sentence></corpus>.

Even on the very small development corpus, the program takes ~15 secs to read it in, of which around 20% are GC time (that's way too much.)

In particular, a lot of data is spending way too much time in DRAG state. This is the profile:

monitoring DRAG culprits. You can see that decodeDocument gets called a lot, and its data is then stalled until the very end of the execution.

Now, I think this should be easily fixed by folding all this decodeDocument stuff into my data structures (Sentence and Word) and then the RT can forget about these thunks. The way it's currently happening though, is that the folding happens at the very end when I force evaluation by deconstruction of Either in the IO monad, where it could easily happen online. I see no reason for this, and my attempts to strictify the program have so far been in vain. I hope somebody can help me :-)

I just can't even figure out too many places to put seqs and $!s in…

Answer 1

One possible thing to try: the default hxt parser is strict, but there does exist a lazy parser based on tagsoup: http://hackage.haskell.org/package/hxt-tagsoup

In understand that expat can do lazy processing as well: http://hackage.haskell.org/package/hxt-expat

You may want to see if switching parsing backends, by itself, solves your issue.