H.264 coded NAL/slice splitting

Question

I'm working on an application that needs to transfer an H.264-encoded video in a real-time over UDP (a sort of a video call).

Recently we've switched to a hardware encoder, which supports only a limited number of H.264 profiles. As the result, each encoded video frame now consists of a single NALu. More precisely, for IDR (key) frames the encoder produces SSP, PSP and a single IDR slice, for others - a single non-IDR slice.

Now, my goal is to split the slice NALu into several smaller ones, because in case of a packet loss if the NALu can't be assembled completely - it's totally lost. At minimum I need to split the slice spatially, i.e. put ranges of macroblocks into different NALus. If possible - I'd also like to extract quality layers, so that the base layer can be protected by more redundant packets (FEC).

Note: I'm not talking about transcoding. It's about repacking/reformatting, locating the macroblock definition data chunks and putting them in a different way.

I'm now trying to parse the appropriate data headers: SPS, PPS and coded slices, using both H.264 standard specs and some open-source decoders code. The task seems possible, though kinda tricky, with a lot of technical details.

My question is: is this a known problem? Is there some API/library that does exactly that?

Answer 1

The question is: is this a known problem? Is there some API/library that does exactly that?"

There is no API library that I know for, but I believe that you can find your solution, or close to your solution here in the source code that could be compiled into the library from the h264bitstream project at github: https://github.com/aizvorski/h264bitstream/blob/master/svc_split.c Let us know how it worked for you. Naturally, you have to do get some code reading here, but the basic solution, or close to the solution seems to be there.