Parsec - error "combinator 'many' is applied to a parser that accepts an empty string"

Question

I'm trying to write a parser using Parsec that will parse literate Haskell files, such as the following:

The classic 'Hello, world' program.

\begin{code}

main = putStrLn "Hello, world"

\end{code}

More text.

I've written the following, sort-of-inspired by the examples in RWH:

import Text.ParserCombinators.Parsec

main
    = do contents <- readFile "hello.lhs"
         let results = parseLiterate contents
         print results

data Element
    = Text String
    | Haskell String
    deriving (Show)


parseLiterate :: String -> Either ParseError [Element]

parseLiterate input
    = parse literateFile "(unknown)" input



literateFile
    = many codeOrProse

codeOrProse
    = code <|> prose

code
    = do eol
         string "\\begin{code}"
         eol
         content <- many anyChar
         eol
         string "\\end{code}"
         eol
         return $ Haskell content

prose
    = do content <- many anyChar
         return $ Text content

eol
    =   try (string "\n\r")
    <|> try (string "\r\n")
    <|> string "\n"
    <|> string "\r"
    <?> "end of line"

Which I hoped would result in something along the lines of:

[Text "The classic 'Hello, world' program.", Haskell "main = putStrLn \"Hello, world\"", Text "More text."]

(allowing for whitespace etc).

This compiles fine, but when run, I get the error:

*** Exception: Text.ParserCombinators.Parsec.Prim.many: combinator 'many' is applied to a parser that accepts an empty string

Can anyone shed any light on this, and possibly help with a solution please?

Answer 1

As sth pointed out many anyChar is the problem. But not just in prose but also in code. The problem with code is, that content <- many anyChar will consume everything: The newlines and the \end{code} tag.

So, you need to have some way to tell the prose and the code apart. An easy (but maybe too naive) way to do so, is to look for backslashes:

literateFile = many codeOrProse <* eof

code = do string "\\begin{code}"
          content <- many $ noneOf "\\"
          string "\\end{code}"
          return $ Haskell content

prose = do content <- many1 $ noneOf "\\"
           return $ Text content

Now, you don't completely have the desired result, because the Haskell part will also contain newlines, but you can filter these out quite easily (given a function filterNewlines you could say `content <- filterNewlines <$> (many $ noneOf "\\")).

Edit

Okay, I think I found a solution (requires the newest Parsec version, because of lookAhead):

import Text.ParserCombinators.Parsec
import Control.Applicative hiding (many, (<|>))

main
    = do contents <- readFile "hello.lhs"
         let results = parseLiterate contents
         print results

data Element
    = Text String
    | Haskell String
    deriving (Show)    

parseLiterate :: String -> Either ParseError [Element]

parseLiterate input
    = parse literateFile "" input

literateFile
    = many codeOrProse

codeOrProse = code <|> prose

code = do string "\\begin{code}\n"
          c <- untilP (string "\\end{code}\n")
          string "\\end{code}\n"
          return $ Haskell c

prose = do t <- untilP $ (string "\\begin{code}\n") <|> (eof >> return "")
           return $ Text t

untilP p = do s <- many $ noneOf "\n"
              newline
              s' <- try (lookAhead p >> return "") <|> untilP p
              return $ s ++ s'

untilP p parses a line, then checks if the beginning of the next line can be successfully parsed by p. If so, it returns the empty string, otherwise it goes on. The lookAhead is needed, because otherwise the begin\end-tags would be consumed and code couldn't recognize them.

I guess it could still be made more concise (i.e. not having to repeat string "\\end{code}\n" inside code).

Answer 2

I haven't tested it, but:

• many anyChar can match an empty string
• Therefore prose can match an empty string
• Therefore codeOrProse can match an empty string
• Therefore literateFile can loop forever, matching infinitely many empty strings

Changing prose to match many1 characters might fix this problem.

(I'm not very familiar with Parsec, but how will prose know how many characters it should match? It might consume the whole input, never giving the code parser a second chance to look for the start of a new code segment. Alternatively it might only match one character in each call, making the many/many1 in it useless.)

Answer 3

For reference, here's another version I came up with (slightly expanded to handle other cases):

import Text.ParserCombinators.Parsec

main
    = do contents <- readFile "test.tex"
         let results = parseLiterate contents
         print results

data Element
    = Text String
    | Haskell String
    | Section String
    deriving (Show)

parseLiterate :: String -> Either ParseError [Element]

parseLiterate input
    = parse literateFile "(unknown)" input

literateFile
    = do es <- many elements
         eof
         return es

elements
    = try section
  <|> try quotedBackslash
  <|> try code
  <|> prose

code
    = do string "\\begin{code}"
         c <- anyChar `manyTill` try (string "\\end{code}")
         return $ Haskell c

quotedBackslash
    = do string "\\\\"
         return $ Text "\\\\"

prose
    = do t <- many1 (noneOf "\\")
         return $ Text t

section
    = do string "\\section{"
         content <- many1 (noneOf "}")
         char '}'
         return $ Section content