How can I force wireshark to decipher some types of protocols if it doesn't recognize them (RTSP,RTP, RTCP)?

Question

I'm trying to implement RTSP protocol in Java according to http://www.csee.umbc.edu/~pmundur/courses/CMSC691C/lab5-kurose-ross.html example; I have succeed up to successful communication with VLC via RTSP requests and streaming RTP packets; RTP packets with JPEG payload are not recognized by VLC well; that's why I supposed I send malformed RTP. I sniffed them with wireshark and compared them with packets, sniffed from successfull RTSP communication of gstreamer RTSP streamer and VLc. I was surprised, that both my app&VLC's RTSP and RTP requests were labeled in wireshark UI as simply TCP and UDP packets, while gstreamer&VLC's one were labeled as RTSP, RTP, RTCP, and even RTSP/SDP.

I'm really confused, I don't understand why VLC's requsts in some cases are parsed by wireshark as pure RTSP/RTP, and im my case as plain TCP/UDP data. At least, VLC's requests are not malformed, even if my are.

How can I force wireshark to recognize that requests as some specific protocol?

Answer 1

Right click on a packet and select "Decode as...". Then choose "RTP" in the list of "Transport" protocols.

If you want to decode some custom-made protocol, you'll have to get or compile a dissector, which can be quite troublesome... Fortunately, PCAP format is simple and plugin directly to it is easy (I happened to just have made one last week, I might make it Open Source if my boss agrees).

Thanks to @GuyHarris who stated that automatic RTP decoding is performed when SDP packets are detected that initiate the RTP session. There also is an option in the "Edit > Preferences" menu, choosing "RTP" under the "Protocols" tree: "Try to decode RTP outside of conversations".