How to build a protocol on top of tcp?

Question

I was searching a lot but I could not find any resources which go through the building of an own protocol which uses TCP as a transport layer. What are the necessary steps? The protocol should be some kind of "Control protocol" for devices. So I can send commands to devices and control them and get information back.

So how would one implement a custom protocol? Is it all about defininig what commands can be sent and how the receiver reacts to different commands? Say I am defining some custom commands with xml send them over the wire/air, using tcp, and have some logic there which reacts to the sent command and replies. Is this a way one could implement a "protocol" ? Is this even called a "protocol"?

kind regards.

Answer 1

As long as you can write a specification that defines the data you send through the TCP socket, you've got your own protocol.

It's mostly about defining commands and payloads. You've got to serialize your command packet before putting them through TCP. Endianness is a common pitfall if you pack the packet in binary format. XML and JSON are common text-based data exchange formats. Personally I'm pro-JSON.

Do refer to BSON, MessagePack or protobuf for binary serialization. They pack the typed data into binary so they are usually more performant, more compact in size and have better type checking than text based serialization. They also handle endian conversion, packet versioning and provide drivers/bindings in various languages. The server and client could have been written in different languages.

EDIT: added RFC samples

Seeing the comment by Ross Patterson, I also recommend reading RFC for protocol definition references. RTSP and HTTP are text protocols, RTP and media formats (MPEG4 AV, H-264) are binary protocols.

EDIT:

Demystifying Protocols and Serialization Performance with Todd Montgomery