How do you write your own IP protocol? (Assuming TCP and UDP are not suitable)

Question

Assuming that you have determined that for a given niche case, neither TCP or UDP are ideal, how would you go about writing your own IP based protocol?

For example, if you're developing on Linux, where would you look in the kernel to "hook" your protocol in?

Where would you start?

Answer 1

You can do this through a kernel module. I would start by reading how arp works for example. That is a simpler protocol since userspace doesn't send packets out with it directly.

The entry point for creating a new network protocol is dev_add_pack, and the code for arp can be found here.

Answer 2

If your protocol can be implemented directly on top of IP, then it can also be implemented wrapped in UDP packets - and the latter has the advantage that it'll pass through existing NAT devices and firewalls that would simply drop your custom protocol.

Answer 3

Read up on UNIX sockets and networking. It's not so much 'hooking' into the kernel, as it is opening a socket and sending your binary data over that.