Is there a way to `read` lazily?

Question

I have probably just spend a day of computation time in vain :)

The problem is that I (naively) wrote about 3.5GB of (compressed) [(Text, HashMap Text Int)] data to a file and at that point my program crashed. Of course there is no final ] at the end of the data and the sheer size of it makes editing it by hand impossible.

The data was formatted via Prelude.show and just at this point I realize that Prelude.read will need to the whole dataset into memory (impossible) before any data is returned.

Now ... is there a way to recover the data without resorting to write a parser manually?

Update 1

main = do
  s <- getContents
  let hs = read s :: [(String, M.Map String Integer)]
  print $ head hs

This I tried ... but it just keeps consuming more memory until it gets killed by the OS.

Answer 1

Sort of. You will still be writing a parser manually... but it is a very short and very easy-to-write parser, because almost all of it will ship out to read. The idea is this: read is strict, but reads, when working on a single element, is lazyish. So we just need to strip out the bits that reads isn't expecting when working on a single element. Here's an example to get you started:

> let s = "[3,4,5," ++ undefined
> reads (drop 1 s) :: [(Int, String)]
[(3,",4,5,*** Exception: Prelude.undefined

I included the undefined at the end as evidence that it is in fact not reading the entire String before producing the parsed 3 at the head of the list.