Streamming a h.264 coded video over UDP

Question

I don't know too much about h.264, but the thing is that I've got this video in h.264 in a mp4 container which I would like to stream over UDP.

My question is simple, is there any tweaks I can maybe do while coding the video so that it comes out fairly tolerant to some "light" packet loss?

I know that compressed video usually has a keyframe every N frames and then in between those it just sends the deltas. I can imagine h.264 should is a lot more complex than that, and so the it might not be just so simple.

To be more precise, I've been making some experiments and realized that just by removing 1024 bytes out of a stream of video, I render it completely "unplayable" from the point of the loss and on.

What I would like is for it to tolerate light losses like that, is it possible?

Thanks

Nelson

Answer 1

It depends on what data you're losing. Some data in the H264 stream is not data that can be lost. For example, if your experiment where you dropped 1024 bytes happened to be the first 1024 bytes you sent, then you probably dropped your Sequence Parameter Set and Picture Parameter Set (SPS/PPS), which is basically the information that tells the decoder how to interpret the incoming information. You also probably can't just drop a random 1024 bytes out of the stream; typically H264 is packetized so that sort of thing wouldn't happen anyway.

So H264 does contain some small pieces of data that really aren't "discardable"; many of the streaming protocols like RTSP make this clear by specifically stating you SHOULD NOT send this sort of information through the lossy transport channel, but instead during the SDP exchange (which happens over TCP).

As far as loss-tolerance goes, H264 does have features that are supposed to aid loss tolerance (ASO, FMO, etc.), but in practice most encoders don't support them. If you can, I'd use x264; their low-latency streaming configuration purportedly works well even with packet loss as high as 20% or more. If your H264 is already encoded, then there's not much to be done. Your video is already encoded and you'll have to stream it as-is.