Show progress of Haskell program

Question

I have list in Haskell with some objects. And I need to find out whether someone of these objects satisfied certain condition. So, I wrote the following:

any (\x -> check x) xs

But the problem is that check operation is very expensive, and the list is quite big. I want to see current progress in runtime, for example 50% (1000/2000 checked).
How I can do this?

Answer 1

Since you want to see the progress of your function (which is a side effect of the function) the most obvious solution is to use monads. So the first thing to do is to make a monadic version of the any function:

anyM :: (Monad m) => (a -> m Bool) -> [a] -> m Bool
anyM _ []        = return False
anyM pred (x:xs) = reduce (pred x) xs
    where reduce acc []     = acc
          reduce acc (x:xs) = do
              condition <- acc
              if condition
                  then return condition
                  else reduce (pred x) xs

The above function anyM is a monadic version of the any function. It allows us to produce side effects in addition to checking whether any item in the given list satisfies the given predicate.

We can use the anyM function to create another function which displays a progress bar as a side effect in addition to executing the any function as follows:

anyVar :: (a -> Bool) -> [a] -> IO Bool
anyVar pred xs = anyM check $ zip [1..] xs
    where check (n,x) = do
            putStrLn $ show n ++ " checked. "
            return $ pred x

Notice that since we don't know the length of the list beforehand we only display the number of items in the list checked. If we know the number of items in the list beforehand then we can display a more informative progress bar:

anyFix :: (a -> Bool) -> Int -> [a] -> IO Bool
anyFix pred length xs = anyM check $ zip [1..] xs
    where check (n,x) = do
            putStrLn $ show (100 * n `div` length) ++ "% (" ++
                show n ++ "/" ++ show length ++ " checked). "
            return $ pred x

Use the anyVar function for infinite lists and for lists whose length you don't know beforehand. Use the anyFix function for finite lists whose length you do know beforehand.

If the list is big and you don't know the length of the list beforehand then the length function will need to traverse the entire list to determine its length. Hence it would be better to use anyVar instead.

Finally to wrap it all this is how you would use the above functions:

main = anyFix (==2000) 2000 [1..2000]

In your case you could do the following instead:

main = anyVar check xs

Hope this answer helped you.

Answer 2

Another way of doing this is using a streaming library like conduit or pipes. Here is some sample code using pipes, which prints a dot every time an element of the list arrives to be checked:

import Pipes
import qualified Pipes.Prelude as P 

bigList :: [Int]
bigList = [1,2,3,4]

check :: Int -> Bool
check = (>3)

main :: IO ()
main = do
    result <- P.any check $ each bigList >-> P.chain (\_ -> putStrLn ".")
    putStrLn . show $ result

(each is a function from the Pipes module.)

Now, if you wanted to show percentages, the P.chain (\_ -> putStrLn ".") part of the pipeline would have to be a bit smarter. It would have to carry the current percentage as state, and know the lenght of the list. (If your list is enormous and lazily generated, calculating its length would force its evaluation and possibly cause problems. If you already have it in memory, it wouldn't be much of a problem.)

Edit: here is an possible extension of the previous code that actually shows percentages:

{-# LANGUAGE FlexibleContexts #-}

import Pipes
import qualified Pipes.Prelude as P
import Data.Function
import Control.Monad.RWS

bigList :: [Int]
bigList = [1,2,3,4]

check :: Int -> Bool
check = (>3)

-- List length is the environment, number of received tasks is the state. 
tracker :: (MonadReader Int m, MonadState Int m, MonadIO m) => Pipe a a m r
tracker = P.chain $ \_ -> do
    progress <- on (/) fromIntegral `liftM` (modify succ >> get) `ap` ask
    liftIO . putStrLn . show $ progress

main :: IO ()
main = do
    (result,()) <- evalRWST (P.any check $ each bigList >-> tracker)
                            (length bigList) -- list length as unchanging environment
                            0 -- initial number of received tasks (the mutable state)
    putStrLn . show $ result

It could be further refined to show only significant percentage increases.

Answer 3

The most naive and direct way is to implement your own

anyM :: (a -> Bool) -> [a] -> IO Bool

that prints the progress bar (e.g. using terminal-progress-bar).

But note that in order to calculate a percentage, you will have to evaluate the full list. This breaks lazyness and can have bad and unwanted effects on the space behaviour of the program.

There are also approaches using unsafePerformIO and unsafeInterleaveIO that allows you to monitor a pure calculation (such as any), see bytestring-progress for an example. But this is dubious design that you should only use if you know that you understand the consequences.

Answer 4

I would just use Debug.Trace.trace and keep track of current position like so:

any (\(i,x) -> trace (showProgress i (length xs)) $ check x) $ zip [1..] xs