Multiplexing streams? (Simultaneous r/w)

Question

I've got a WebM writer instance that takes video frames as byte arrays (media buffers) from a VP8 encoder instance and continuously writes them into a MemoryStream while packing them up into a WebM container.

I also got a HttpListener which asynchronously waits for http requests. When a client arrives, a new HttpListenerContext gets created which holds a Stream instance to write in the response stream (Context.Response.OutputStream). Now I would like to 'multiplex' my memory stream, e.g. continously reading from it and sharing out to the different connected http clients. So:

1. Take frames from VP8 encoder
2. Write them into a memory stream using WebM writer

-> Here I'm stuck

3. Simultaneously read from the memory stream
4. Continously write the data into the response stream of connected http clients

The whole task is for a HTTP live streaming server. Unfortunately, I can't show the source of the WebM writer/VP8 encoder etc. because proprietary components, but I hope you get the idea.

I already tried using stream.CopyTo(), but it seems like the HttpListenerContext understands the copy operation as finishing, thus returning only a chunk of what should be sent. Also reading from the stream using a StreamReader sets the position forward with each operation, so this is also useless.

Answer 1

Like Jim's answer, I have written Nerdbank.FullDuplexStream which sounds like it may be what you want.

Answer 2

For #3, you need a stream that acts like a FIFO queue. The .NET Framework doesn't provide such a stream. However, I created one a few years ago that does exactly what you're looking for. It lets one thread read while the other is writing. You can find full source, an explanation of how it works, and an example at Building a new type of stream.

Basically, it's just a big memory buffer that I treat like a circular queue. If the buffer fills up, then the writer blocks until some data is read. And if there's no data to read, the reader blocks until some data is written or until the stream is marked as done (i.e. end of stream).

Another example is in this answer.

Solving #4 in your list is just a matter of a loop that writes to each response stream.