What happens if an Enumerator tries to consume input?

Question

The definition of Enumerator is:

type Enumerator a m b = Step a m b -> Iteratee a m b

The documentation states that while Iteratees comsume data, Enumerators produce it. I can understand how one might produce data with such a type:

enumStream :: (Monad m) => Stream a -> Enumerator a m b
enumStream stream step =
    case step of
        Continue k -> k stream
        _          -> returnI step  -- Note: 'stream' is discarded

_{(enumEOF is more complicated than this... it apparently checks to make sure the Iteratee does not Continue after being given EOF, throwing an error if it does.)}

Namely, an Iteratee produces a Step when it is run with runIteratee. This Step is then fed to my enumerator, which supplies it with a Stream so it can continue. My enumerator returns the resulting continuation.

One thing stands out at me: this code is running in the Iteratee monad. That means it can consume data, right?

-- | Like 'enumStream', but consume and discard a chunk from the input stream
--   simply because we can.
enumStreamWeird :: (Monad m) => Stream a -> Enumerator a m b
enumStreamWeird stream step = do
    _ <- continue return            -- Look, mommy, I'm consuming input!
    case step of
        Continue k -> k stream
        _          -> returnI step

The documentation states that when an enumerator acts as both a source and sink, Enumeratee should be used instead:

type Enumeratee ao ai m b = Step ai m b -> Iteratee ao m (Step ai m b)

However, apparently I didn't have to; I could consume input in the definition of an Enumerator, as demonstrated by my enumStreamWeird function.

My questions are:

• What happens if you try to "consume" data within an Enumerator, like enumStreamWeird does? Where does the data come from?

• Even if we aren't insane enough to consume data in an enumerator, is it valid to perform actions in the underlying monad on the behalf of the enumerator, rather than on behalf of the iteratee reading the data we're producing?

The latter question might be less related to my main question, but I'm trying to understand how an Enumerator does what it does.

Answer 1

Yes, an enumerator can consume data. An enumerator basically takes an iteratee and transforms it into the same iteratee after it has been fed some items. If the enumerator asks for input, then the resulting iteratee will ask for input.

How an Enumerator is fed to an Iteratee

Let's look at how an enumerator is fed to an iteratee:

-- | Feed an Enumerator to an Iteratee.
feed :: Monad m
     => Iteratee a m b
     -> Enumerator a m b
     -> Iteratee a m b
feed iteratee enumerator =
    Iteratee $ do
        step <- runIteratee iteratee
        runIteratee $ enumerator step

_{Note: feed is a special case of >>==.}

First, feed runs the iteratee until it is ready for input. Then, it passes the iteratee's first Step to the enumerator. The enumerator takes over from there.

That last sentence is very important. The enumerator can do whatever it wants with its iteratee. It can discard the iteratee entirely if it wants to. However, an enumerator usually supplies the iteratee with the input it has, then hands control back to the iteratee.

Example 1: Feeding enumerators to an iteratee

Suppose we have an iteratee that asks for three strings and prints them:

iter3 :: Iteratee String IO ()
iter3 = do
    lift $ putStrLn "Gimmie a string!"
    a <- head_
    lift $ putStrLn a
    lift $ putStrLn "Gimmie another string!"
    b <- head_
    lift $ putStrLn b
    lift $ putStrLn "Gimmie one more string!"
    c <- head_
    lift $ putStrLn c
    lift $ putStrLn "Thank you!"

_{head_ is defined in Data.Enumerator.List.}

and an enumerator that feeds its iteratee a single string:

getString :: Enumerator String IO a
getString (Continue k) = do
    line <- lift getLine
    k (Chunks [line])
getString step = Iteratee $ return step

When getString is given an iteratee that needs more than one item, it will feed the iteratee with the first item. Then, getString itself will need the remaining items.

• iter3 needs three items before it can return ().

• iter3 `feed` getString needs two items.

• iter3 `feed` getString `feed` getString needs one item.

• iter3 `feed` getString `feed` getString `feed` getString does not need any more items.

• iter3 `feed` getString `feed` getString `feed` getString `feed` getString is equivalent to the above. This is handled by getString's second case.

Example 2: An enumerator that consumes input

Consider an enumerator that does consume input:

consumeEnum :: Enumerator String IO a
consumeEnum step = do
    lift $ putStrLn "I take without giving"
    _ <- head_
    Iteratee $ return step

What does iter3 `feed` consumeEnum do? That can sort of be answered by looking at consumeEnum's own implementation. First it needs an item and discards it. Then it hands the torch to iter3, which needs three more items.

However, look back to the feed combinator. It starts by running iter3, then passes its Step to consumeEnum. This means "Gimmie a string!" will be printed before control reaches consumeEnum.

Answer 2

There is nothing wrong with an enumerator consuming data. It is an iteratee transformer, which may well feed its own input into its iteratee. Look at the way you apply an enumerator to an iteratee. You can also apply another enumerator to an iteratee applied to an enumerator.