Prevent kernel from processing TCP segments bound to a raw socket

Question

According to http://linux.die.net/man/7/raw , raw_socket = socket(AF_INET, SOCK_RAW, int protocol); is the way to create a raw socket.

1. I assume that raw-sockets are created on layer-3 and so protocol shouldn't be IPPROTO_TCP / IPPROTO_UDP but it should be IPPROTO_IP. Is this understanding correct?

2. But when I create the raw socket with protocol as IPPROTO_IP (*socketFd = socket(AF_INET, SOCK_RAW, IPPROTO_IP);), socket creation fails with the error Protocol not supported

3. When I create the raw socket with protocol as IPPROTO_RAW (*socketFd = socket(AF_INET, SOCK_RAW, IPPROTO_RAW);), my application doesn't receive any of packets

4. When I create the raw socket with protocol as IPPROTO_TCP (socketFd = socket(AF_INET, SOCK_RAW, IPPROTO_TCP);), my application receives the TCP packets, but kernel also responds to these packets (and in my case it RSTs the link). I assume it is because kernel thinks there isn't anybody listening to the port to which that packet is intended to.

My intention is just to send responses to the messages coming to my application with a fake IP and TCP header. Since none of the above tries worked for me, how should I create the raw socket and make kernel TCP layer to be quiet for only that connection?

EDIT: Please skip questions 1-3. They are already answered by Filipe. For ques 4, we do have a workaround. But keeping the question open, if someone out here has an answer and would like to answer it.

Answer 1

I assume that raw sockets are created on layer-3 and so protocol shouldn't be IPPROTO_TCP / IPPROTO_UDP but it should be IPPROTO_IP. Is this understanding correct?

No. You are right that raw sockets are basically layer 3 packets, but the protocol should not be IPPROTO_IP. The protocol argument in the case of raw sockets indicates what type of packets you are interested in receiving on that socket. Remember that a protocol essentially performs transport-level demultiplexing, so you need to specify what type of protocol your raw socket is interested in. This is made clear in man 7 raw:

All packets or errors matching the protocol number specified for the raw socket are passed to this socket. For a list of the allowed protocols see RFC 1700 assigned numbers and getprotobyname(3).

Since you are interested in receiving IP packets for a TCP connection, you should use IPPROTO_TCP.

But when I create the raw socket with protocol as IPPROTO_IP (*socketFd = socket(AF_INET, SOCK_RAW, IPPROTO_IP);), socket creation fails with the error Protocol not supported.

Yes, that's kind of expectable: the IP protocol is not a layer 4 protocol. As I said, the protocol field is used for transport-layer demultiplexing, so it makes little sense to use IPPROTO_IP.

When I create the raw socket with protocol as IPPROTO_RAW (*socketFd = socket(AF_INET, SOCK_RAW, IPPROTO_RAW);), my application doesn't receive any of packets

That's because IPPROTO_RAW means that you are interested in sending all types of protocol packets (TCP, UDP, or any other protocol). But with IPPROTO_RAW you can't do the opposite: IPPROTO_RAW would mean that you could receive any protocol in this raw socket, which is not supported. This is also made clear in man 7 raw:

A protocol of IPPROTO_RAW implies enabled IP_HDRINCL and is able to send any IP protocol that is specified in the passed header. Receiving of all IP protocols via IPPROTO_RAW is not possible using raw sockets.

In other words, IPPROTO_RAW gives you the power to send packets matching any protocol, but at the cost of preventing you from ever getting a reply. You could create other specific raw sockets tied to a protocol to get the replies as a workaround, but this complicates the design because you'd have to manage a pool of raw sockets, and it is most definitely not what you want to do here.

When I create the raw socket with protocol as IPPROTO_TCP (socketFd = socket(AF_INET, SOCK_RAW, IPPROTO_TCP);), my application receives the TCP packets, but kernel also responds to these packets (and in my case it RSTs the link). I assume it is because kernel thinks there isn't anybody listening to the port to which that packet is intended to.

You can't prevent the kernel from doing its job. From the raw sockets manpage:

When a packet is received, it is passed to any raw sockets which have been bound to its protocol before it is passed to other protocol handlers (e.g., kernel protocol modules).

So you are right that the kernel sends an RST packet because it has no knowledge of active TCP sockets or connections on the specified port. As I said, you can't stop the kernel from doing its work, but a relatively quick (and perhaps ugly) hack is to drop RST packets with iptables:

iptables -A OUTPUT -p tcp --tcp-flags RST RST -j DROP

Yes, not very elegant, but I think there's not much we can do here.

As suggested in the comments, you might also create a dummy TCP socket bound to the same port and address where you just receive and discard the messages. That way the kernel won't send RST replies, and you don't need to mess with iptables.

Also remember that since you need to specify IPPROTO_TCP for your raw socket, you should set IP_HDRINCL on the socket with setsockopts(2) so that you can build the custom IP header.

Finally, make sure the process running this has an effective user ID of 0 or the CAP_NET_RAW capability (in practice: run it as root).