How to do non blocking socket reads with Protobuf using C#?

Question

Lets say I want to do non blocking reads from a network socket. I can async await for the socket to read x bytes and all is fine.

But how do I combine this with deserialization via protobuf?

Reading objects from a stream must be blocking? that is, if the stream contains too little data for the parser, then there has to be some blocking going on behind the scenes so that the reader can fetch all the bytes it needs.

I guess I can use lengthprefix delimiters and read the first bytes and then figure out how many bytes I have to fetch minimum before I parse, is this the right way to go about it? e.g. if my buffer is 500 bytes, then await those 500 bytes, and parse the lengthprefix and if the length is more than 500 then wait again, untill all of it is read.

What is the idiomatic way to combine non blocking IO and protobuf parsing?

(I'm using Jon Skeet's implementation right now http://code.google.com/p/protobuf-csharp-port/)

Answer 1

As a general rule, serializers don't often contain a DeserializeAsync method, because that is really really hard to do (at least, efficiently). If the data is of moderate size, then I would advise to buffer the required amount of data using asynchronous code - and then deserialize when all of the required data is available. If the data is very large and you don't want to have to buffer everything in memory, then consider using a regular synchronous deserialize on a worker thread.

(note that note of this is implementation specific, but if the serializer you are using does support an async deserialize: then sure, use that)