Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Ensuring files are closed promptly

I am writing a daemon that reads something from a small file, modifies it, and writes it back to the same file. I need to make sure that each file is closed promptly after reading before I try to write to it. I also need to make sure each file is closed promptly after writing, because I might occasionally read from it again right away.

I have looked into using binary-strict instead of binary, but it seems that only provides a strict Get, not a strict Put. Same issue with System.IO.Strict. And from reading the binary-strict documentation, I'm not sure it really solves my problem of ensuring that files are promptly closed. What's the best way to handle this? DeepSeq?

Here's a highly simplified example that will give you an idea of the structure of my application. This example terminates with

*** Exception: test.dat: openBinaryFile: resource busy (file is locked)

for obvious reasons.

import Data.Binary ( Binary, encode, decode )
import Data.ByteString.Lazy as B ( readFile, writeFile )
import Codec.Compression.GZip ( compress, decompress )

encodeAndCompressFile :: Binary a => FilePath -> a -> IO ()
encodeAndCompressFile f = B.writeFile f . compress . encode

decodeAndDecompressFile :: Binary a => FilePath -> IO a
decodeAndDecompressFile f = return . decode . decompress =<< B.readFile f

main = do
  let i = 0 :: Int
  encodeAndCompressFile "test.dat" i
  doStuff

doStuff = do
  i <- decodeAndDecompressFile "test.dat" :: IO Int
  print i
  encodeAndCompressFile "test.dat" (i+1)
  doStuff
like image 789
mhwombat Avatar asked May 09 '12 14:05

mhwombat


2 Answers

All 'puts' or 'writes' to files are strict. The act of writeFile demands all Haskell data be evaluated in order to put it on disk.

So what you need to concentrate on is the lazy reading of the input. In your example above you both lazily read the file, then lazily decode it.

Instead, try reading the file strictly (e.g. with strict bytestrings), and you'll be fine.

like image 73
Don Stewart Avatar answered Sep 21 '22 06:09

Don Stewart


Consider using a package such as conduit, pipes, iteratee or enumerator. They provide much of the benefits of lazy IO (simpler code, potentially smaller memory footprint) without the lazy IO. Here's an example using conduit and cereal:

import Data.Conduit
import Data.Conduit.Binary (sinkFile, sourceFile)
import Data.Conduit.Cereal (sinkGet, sourcePut)
import Data.Conduit.Zlib (gzip, ungzip)
import Data.Serialize (Serialize, get, put)

encodeAndCompressFile :: Serialize a => FilePath -> a -> IO ()
encodeAndCompressFile f v =
  runResourceT $ sourcePut (put v) $$ gzip =$ sinkFile f

decodeAndDecompressFile :: Serialize a => FilePath -> IO a
decodeAndDecompressFile f = do
  val <- runResourceT $ sourceFile f $$ ungzip =$ sinkGet get
  case val of
    Right v  -> return v
    Left err -> fail err

main = do
  let i = 0 :: Int
  encodeAndCompressFile "test.dat" i
  doStuff

doStuff = do
  i <- decodeAndDecompressFile "test.dat" :: IO Int
  print i
  encodeAndCompressFile "test.dat" (i+1)
  doStuff
like image 24
Nathan Howell Avatar answered Sep 22 '22 06:09

Nathan Howell