Multiple H.264 video streams in one RTP session

Question

I would like to dynamically switch the video source in a streaming video application. However, the different video sources have unique image dimensions. I can generate individual SDP files for each video source, but I would like to combine them into a single SDP file so that the viewing client could automatically resize the display window as the video source changed. Here are two example SDP files:

640x480.sdp:

v=0
o=VideoServer 305419896 9876543210 IN IP4 192.168.0.2
s=VideoStream640x480
t=0 0
c=IN IP4 192.168.0.2
m=video 8000/2 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=0; profile-level-id=4D4033; sprop-parameter-sets=Z01AM5ZkBQHtCAAAAwAIAAADAYR4wZU=,aO48gJ==
a=control:trackID=1

960x480.sdp:

v=0
o=VideoServer 305419896 9876543210 IN IP4 192.168.0.2
s=VideoStream960x480
t=0 0
c=IN IP4 192.168.0.2
m=video 8000/2 RTP/AVP 96
a=rtpmap:96 H264/90000
a=fmtp:96 packetization-mode=0; profile-level-id=4D4033; sprop-parameter-sets=J01AM5WwPA9sBAIA,KO4G8gA=
a=control:trackID=1

How can these individual files be combined into a single SDP file?

Answer 1

The parameters in your two sdp examples are very close - the stream name and the sprop-parameter-sets differ. I assume you don't care about the stream name. If you need separate sprop-parameter-sets and the clients support the standard well you can use separate dynamic payload types for each resolution and have a single SDP as follows:

    v=0
    o=VideoServer 305419896 9876543210 IN IP4 192.168.0.2
    s=VideoStream640x480
    t=0 0
    c=IN IP4 192.168.0.2
    m=video 8000/2 RTP/AVP 96 97
    a=rtpmap:96 H264/90000
    a=fmtp:96 packetization-mode=0; profile-level-id=4D4033; sprop-parameter-sets=Z01AM5ZkBQHtCAAAAwAIAAADAYR4wZU=,aO48gJ==
    a=rtpmap:97 H264/90000
    a=fmtp:97 packetization-mode=0; profile-level-id=4D4033; sprop-parameter-sets=J01AM5WwPA9sBAIA,KO4G8gA=
    a=control:trackID=1

Similar to other answers if you don't actually need the different stream names or the different sprop-parameter-sets you should be able to use your first SDP and switch format mid stream. I don't know the actual payload of H.264 or your particular decoder well enough to ensure that this will work in your applications but it is very common in videoconferencing applications to allow dynamically switching between resolutions without signaling a change or requiring a separate dynamic payload type.

Although you can concatenate two SDP documents as mentioned in another answer I don't think it will help in this case. H.264 decoders can only work with a single sprop-parameter-sets parameter at a time I believe. Since both SDPs would have the same payload type, source port, etc. the receiver would not know when to use which sprop-parameter-sets parameter. UPDATE: Note some implementations get their sprops inband and not from the SDP (or only initially from the SDP). If that applies in your environment the SDP sprop-parameter-sets can be updated inband

References:

1. RFC 3984 RTP Payload Format for H.264 Video
2. New proposed H.264 RTP Payload Format RFC 6184
3. RFC 4566 SDP: Session Description Protocol

[Sorry for not giving the full cite - feel free to correct]

Answer 2

I've gone over the RFC (RFC2327 - SDP: Session Description Protocol) and it appears you can just concatenate the two SDP documents. The document states explicitly:

When SDP is conveyed by SAP, only one session description is allowed per packet. When SDP is conveyed by other means, many SDP session descriptions may be concatenated together (the `v=' line indicating the start of a session description terminates the previous description).