Live FLV streaming in C# WebApi

Question

Currently I have a working live stream using webapi. By receiving a flv stream directly from ffmpeg and sending it straight to the client using PushStreamContent. This works perfectly fine if the webpage is already open when the stream starts. The issue is when I open another page or refresh this page you can no longer view the stream (the stream is still being sent to the client fine). I think it is due to something missing from the start of the stream but I am not sure what to do. Any pointers would be greatly appreciated.

Code for client reading stream

public class VideosController : ApiController
{
    public HttpResponseMessage Get()
    {
        var response = Request.CreateResponse();
        response.Content = new PushStreamContent(WriteToStream, new MediaTypeHeaderValue("video/x-flv"));

        return response;
    }

    private async Task WriteToStream( Stream arg1, HttpContent arg2, TransportContext arg3 )
    {
        //I think metadata needs to be written here but not sure how
        Startup.AddSubscriber( arg1 );
        await Task.Yield();
    }
}

Code for receiving stream and then sending to client

while (true)
{
    bytes = new byte[8024000];
    int bytesRec = handler.Receive(bytes);

    foreach (var subscriber in Startup.Subscribers.ToList())
    {
        var theSubscriber = subscriber;
        try
        {
            await theSubscriber.WriteAsync( bytes, 0, bytesRec );
        }
        catch
        {
            Startup.Subscribers.Remove(theSubscriber);
        }
    }
}

Answer 1

I've never used FLV or studied video formats closely

Most file formats are structured, especially video formats. They contain frames (i.e. a complete or partial screen shots depending on the compression format).

You should be really lucky if you manage to hit a specific frame when you start streaming to the new subscriber. Hence when they start receiving the stream they cannot identify the format as frame is partial.

You can read more FLV frames in wikipedia article. This is most likely your problem.

A simple attempt would be to try to save the initial header that you receive from the streaming server when the first subscriber connects.

Something like:

static byte _header = new byte[9]; //signature, version, flags, headerSize

public void YourStreamMethod()
{
    int bytesRec = handler.Receive(bytes);
    if (!_headerIsStored)
    {
        //store header
        Buffer.BlockCopy(bytes, 0, _header, 0, 9);
        _headerIsStored = true;
    }
}

.. which allows you to send the header to the next connecting subscriber:

private async Task WriteToStream( Stream arg1, HttpContent arg2, TransportContext arg3 )
{
    // send the FLV header
    arg1.Write(_header, 0, 9);

    Startup.AddSubscriber( arg1 );
    await Task.Yield();
}

Once done, pray that the receiver will ignore partial frames. If it doesn't you need to analyze the stream to identify where the next frame is.

To do that you need to do something like this:

1. Create a BytesLeftToNextFrame variable.
2. Store the received packet header in a buffer
3. Convert the "Payload size" bits to an int
4. Reset the BytesLeftToNextFrame to the parsed value
5. Countdown until the next time you should read a header.

Finally, when a new client connects, do not start streaming until you know that the next frame arrives.

Pseudo code:

var bytesLeftToNextFrame = 0;
while (true)
{
    bytes = new byte[8024000];
    int bytesRec = handler.Receive(bytes);

    foreach (var subscriber in Startup.Subscribers.ToList())
    {
        var theSubscriber = subscriber;
        try
        {
            if (subscriber.IsNew && bytesLeftToNextFrame < bytesRec)
            {
                //start from the index where the new frame starts
                await theSubscriber.WriteAsync( bytes, bytesLeftToNextFrame, bytesRec - bytesLeftToNextFrame);
                subscriber.IsNew = false;
            }
            else
            {
                //send everything, since we've already in streaming mode
                await theSubscriber.WriteAsync( bytes, 0, bytesRec );
            }
        }
        catch
        {
            Startup.Subscribers.Remove(theSubscriber);
        }
    }

    //TODO: check if the current frame is done
    // then parse the next header and reset the counter.
}

Answer 2

I'm not a expert in streaming, but looks like you should close stream then all data will be writed

await theSubscriber.WriteAsync( bytes, 0, bytesRec );

Like it mentions in WebAPI StreamContent vs PushStreamContent

{
     // After save we close the stream to signal that we are done writing.
     xDoc.Save(stream);
     stream.Close();
}