I'm trying to write a client for a binary network protocol. All network operations are carried out over a single TCP connection, so in that sense the input from the server is a continuous stream of bytes. At the application layer, however, the server conceptually sends a packet on the stream, and the client keeps reading until it knows the packet has been received in its entirety, before sending a response of its own.
A lot of the effort needed to make this work involves parsing and generating binary data, for which I'm using the Data.Serialize module.
The server sends me a "packet" on the TCP stream. The packet is not necessarily terminated by a newline, nor is it of a predetermined size. It does consist of a predetermined number of fields, and fields generally begin with a 4 byte number describing the length of that field. With some help from Data.Serialize, I already have the code to parse a ByteString version of this packet into a more manageable type.
I'd love to be able to write some code with these properties:
So in brief, is it possible to leverage my current ByteString parsing code in combination with lazy IO to read exactly the right number of bytes off the network?
I tried to use lazy ByteStreams in combination with my Data.Serialize instance, like so:
import Network
import System.IO
import qualified Data.ByteString.Lazy as L
import Data.Serialize
data MyType
instance Serialize MyType
main = withSocketsDo $ do
h <- connectTo server port
hSetBuffering h NoBuffering
inputStream <- L.hGetContents h
let Right parsed = decodeLazy inputStream :: Either String MyType
-- Then use parsed to form my own response, then wait for the server reply...
This seems to fail mostly on point 3 above: it stays blocked even after a sufficient
number of bytes have arrived to parse MyType. I strongly suspect this is because
ByteStrings are read with a given block size at a time, and L.hGetContents
is
waiting for the rest of this block to arrive. While this property of reading an
efficient blocksize is helpful for making efficient reads from disk, it seems to be
getting in my way for reading just enough bytes to parse my data.
Something is wrong with your parser, it is too eager. Most likely it need the next byte after the message for some reason. hGetContents
from bytestring
doesn't block waiting for the whole chunk. It uses hGetSome
internally.
I created simple test case. The server sends "hello" every second:
import Control.Concurrent
import System.IO
import Network
port :: Int
port = 1234
main :: IO ()
main = withSocketsDo $ do
s <- listenOn $ PortNumber $ fromIntegral port
(h, _, _) <- accept s
let loop :: Int -> IO ()
loop 0 = return ()
loop i = do
hPutStr h "hello"
threadDelay 1000000
loop $ i - 1
loop 5
sClose s
The client reads the whole contents lazily:
import qualified Data.ByteString.Lazy as BSL
import System.IO
import Network
port :: Int
port = 1234
main :: IO ()
main = withSocketsDo $ do
h <- connectTo "localhost" $ PortNumber $ fromIntegral port
bs <- BSL.hGetContents h
BSL.putStrLn bs
hClose h
If you try to run both of then, you'll see the client printing "hello" every seconds. So, the network subsystem is ok, the issue is somewhere else -- most likely in your parser.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With