Multiple Ethernet Interfaces - How to create a separate network and access from C code

Question

I have a Linux device (actually a BeagleBoard for the prototype) with two Ethernet adapters. What I want is this:

• Primary ethernet interface (eth0) connects to a client's network (may be DHCP or assigned a static IP).

• Second ethernet interface (eth1) connects directly to another Linux board.

• User-level C programs on the Beagle can listen for incoming connections from the client's network (on eth0), respond as required, and possibly connect to the other device on eth1

• We want the second device to be completely hidden from the rest of the network, so that only programs on the Beagle can connect to it.

I was hoping that the two interfaces could be completely separate, and my code could choose which interface it wanted to listen / open connections on. Typical code:

socket_desc = socket(AF_INET, SOCK_STREAM, 0);

memset(&client_addr, 0, sizeof(client_addr)); 
client_addr.sin_family = AF_INET;
inet_pton(  AF_INET, address, (void *)(&(client_addr.sin_addr.s_addr))  );
client_addr.sin_port = htons(port);   

/* Connect to remote on TCP port: */
connect(*socket_desc, (struct sockaddr*) &client_addr, sizeof(client_addr) );
...

We can set the address and port to connect, but not the ethernet interface. I found this answer. Does this mean that what I am trying to achieve MUST be left to the kernel routing?

I know I can set up eth1 to be on its own subnet, but I have a problem with this: Given that we don't know anything about the client's network, how do I know that whatever subnet I use won't end up conflicting with the client's network? For example, I use 192.168.1.0 here, so I could put eth1 on 192.168.0.0... but what if the client uses that range? We don't want to reconfigure the settings for eth1 and its attached device for every deployment, although we may have to configure eth0.

So is there a better way to do this? Or perhaps a reserved IP address range I could use for the subnet on eth1 which is guaranteed not to conflict with the client's network (e.g. 169.254.1.x)?

Sorry this is a bit vague, but I have been Googling this for days and have probably become more confused rather than less.

EDIT - 2013-08-29:

I've just found this question which provides a partial answer: You can use setsockopt(...) with SO_BINDTODEVICE to bind to a specific device / interface. This seems to work, although I still can't figure out how to set up routing tables to work reliably in the situation where the "internal" network happens to have the same IP address range as the "external" network connected to eth0.

Answer 1

I had the same requirement for a project and we ended up using LXC, where we could assign a physical interface to a container, and so we could have containers with same ip address subnets, for example.

Check it out at http://lxc.sourceforge.net/.