creating my own MJPEG stream

Question

I'm trying to create an MJPEG stream, I have a series of jpegs that I want to put together into a stream so that a user can just hit a URL and get an mjpeg stream. I've been trying for the last few days to get this to work, and it may just not be possible. I've brought up ethereal and listened to the packets coming from an axis camera on the net somewhere, and tried to mimmick it. I originally tried using WCF, and returning a "stream" but then later found out that I would need to set the content type on that stream, so I then tried the WCF REST api, but that suffers from the same problem. so I am now just using a bare bones HTTPListener, and handling the event. I would greatly prefer to use WCF, but I'm not sure that it will allow me to return a stream with the right content type. so here's what I have for the httpListener .

in the handler of the listener call back I put the following.

        HttpListenerResponse response = context.Response;
        response.ProtocolVersion = new System.Version(1, 0);
        response.StatusCode = 200;
        response.StatusDescription = "OK";
        response.ContentType = "multipart/x-mixed-replace;boundary=" + BOUNDARY + "\r\n";
        System.IO.Stream output = response.OutputStream;
        Render(output);

the Render method looks like this

        var writer = new StreamWriter(st);
        writer.Write("--" + BOUNDARY + "\r\n");
        while (true)
        {
            for (int i = 0; i < imageset.Length; i++)
            {
                var resource = Properties.Resources.ResourceManager.GetObject(imageset[i]) as Bitmap;
                var memStream = new MemoryStream();
                resource.Save(memStream,ImageFormat.Jpeg);
                byte[] imgBinaryData = memStream.ToArray();
                string s = Convert.ToBase64String(imgBinaryData);
                writer.Write("Content-type: image/jpeg\r\n");
                foreach (var s1 in imgBinaryData)
                {
                    writer.Write((char)s1);
                }
                writer.Write("\n--" + BOUNDARY + "\n");
                writer.Flush();
                Thread.Sleep(500);
            }
        }

At this point I've just added a few jpeg images as properties on the dll, and am iterating over them, eventually these will be dynamic images, but for now I just want to get the thing to work.

From what I understand about the MJPEG (spec) is that the content must be set to multipart/x-mixed-replace and a boundary set. and then you just deliminate the jpegs within the stream by the boundary.

This seems like it should be simpler then I'm making it, but I'm wondering where I'm going wrong. if I load this URL up in IE or Firefox, it just hangs. if I try to make a stub html page with an img tag, whose source is the URL then I get a broken image.

Any ideas, thanks

Josh

Answer 1

Well, as far as I can tell, here are your issues:

1. The StreamWriter is not a correct choice. Use a regular stream write function is fine. Meaning, you should write data in Byte array instead of string.

2. You convert the Binary data of the image to String64, the browser does not known that, still thinking it is 32bit data.

3. Your jpeg frame format is not correct. You should also add Content-Length to the frame header so that the application that receive the stream know when to stop reading rather than having to check for the next boundary string every read. This will result in about 4-5 times faster in reading data. And there are also inconsistency in your new line character, some are "\r\n" while some others are "\n".

4. While loop is a infinite loop.

So, here is the solution.

Note: There might be some syntax errors but you probably get the general idea.

private byte[] CreateHeader(int length)
{
    string header = 
        "--" + BOUDARY + "\r\n" +
        "Content-Type:image/jpeg\r\n" +
        "Content-Length:" + length + "\r\n" +
        + "\r\n"; // there are always 2 new line character before the actual data

    // using ascii encoder is fine since there is no international character used in this string.
    return ASCIIEncoding.ASCII.GetBytes(header); 
}

public byte[] CreateFooter()
{
    return ASCIIEncoding.ASCII.GetBytes("\r\n");
}

private void WriteFrame(Stream st, Bitmap image)
{
    // prepare image data
    byte[] imageData = null;

    // this is to make sure memory stream is disposed after using
    using (MemoryStream ms = new MemoryStream())
    {
        image.Save(ms, ImageFormat.Jpeg);

        imageData = ms.ToArray();
    }

    // prepare header
    byte[] header = CreateHeader(imageData.Length);
    // prepare footer
    byte[] footer = CreateFooter();

    // Start writing data
    st.Write(header, 0, header.Length);
    st.Write(imageData, 0, imageData.Length);
    st.Write(footer, 0, footer.Length);
}

private void Render(Stream st)
{
    for (int i = 0; i < imageset.Length; i++)
    {
        var resource = Properties.Resources.ResourceManager.GetObject(imageset[i]) as Bitmap;
        WriteFrame(st, resource);
        Thread.Sleep(500);
    }
}