How to identify a specific socket between User Space and Kernel Space?

Question

I have a library in User Space that intercepts socket layer calls such as socket(), connect(), accept(), etc. I'm only dealing with TCP sockets.

Down in Kernel Space I have a network kernel module, which deals with all the TCP connections. I need to be able to identify in the driver which sockets were intercepted by the User Space library.

So far I've been using the priority field from struct sock (Kernel) that I can set with setsockopt() in User Space. But that's quite a dirty hack.

Is there any kind of private field of struct sock I could safely use and set from User Space through setsockopt()?

Thanks.

Answer 1

There is really no such "private field" option that can be used solely by user space and your kernel code.

Using the SO_PRIORITY option seems a little too intrusive, as it can change how the stack processes packets, and it might lead to hard to understand results. A safer option would be to adjust the SO_RCVBUF or SO_SNDBUF values by some small delta from the usual default. Linux will double the value passed in, but you can look for the delta from the default values and know that the presence of the delta as a signal that this is your "intercepted" socket.

Answer 2

Getting to your original question "I need to be able to identify in the driver which sockets were intercepted by the User Space library." there are a few functions in fact.

Firstly you need to know that ALL the existing connections are stored in a global hash table - "tcp_hashinfo", and you can find the address in /proc/kallsyms.

The main function is __inet_lookup_skb(), which called __inet_lookup(), and split into __inet_lookup_established() (looking for any matching sockets that have been established) and __inet_lookup_listener() (looking for opened listener, but no established connections yet).

The main inputs required for lookup is the source/destination ports and IP addresses information, if found the returning pointer is a "struct sock *" to the socket.

IPv6 has about the same name, and same logic too.

The function (__inet_lookup_skb()) is declared "static inline" - it cannot be found from /proc/kallsyms and neither can drivers see it, as it is compiled inline. But no problem for that as this call two other function - inet_lookup_listener() and inet_lookup_established() which are NOT compiled inline. Symbols for it are exported, so you are safe to use it from a kernel module.

It is important that reading this hashinfo table is a critical operation - multiple CPUs may be reading/writing to it at the same time, and which is why locking is done at the beginning/end of functions when reading is done, to be prevent it being modified while being read. As it is difficult to access these RCU locks, even though it is global by nature, don't re-implement these functions, just reuse them.