How to detect specific devices in a local network?

Question

I am building an IoT solution where I will have multiple devices connected to my local network that I want to connect to a hub that controls my solution.

Once I connect a new device I want to connect it with the hub and give it a name to identify it. I would like to implement the detection in an automatic way (so I don´t have to enter IP addresses manually). As an example when a Chromecast is present in a network I can see it in my streaming applications in my phone. So I want to do something similar to connect the hub with the devices.

My ideas so far is that it is two ways of doing this:

1. The hub scans the network for new devices (either periodically or when I say there is a new device present).
2. The devices scan the network to find the hub once connected.

Is any of these approaches to prefer over the other and in that case why?

When doing a scan, in whatever direction I choose, what is the most effective way to do the scan? I am doing an implementation using Java and what I have so far is this:

int timeout = 100;
for (int i = 1; i < 255; i++)
{
    String host = subnet + "." + i;

    if (InetAddress.getByName(host).isReachable(timeout))
    {
        String hostname = InetAddress.getByName(host).getHostName();
        String canonicalHostName = InetAddress.getByName(host).getCanonicalHostName();
        System.out.println(host + " is reachable. Hostname: " + hostname + ", CanonicalHostName: " + canonicalHostName);
    }
}

What I have seen here is that the hostname returned is for most stuff in my network just the IP address, and not the name that I see as the hostname in my router. I thought I could use the hostname as an identifyer to detect specific devices and understand what they where - but with this small poc that does not seem to work. So how can I identify the devices in a good way?

Is there any library/solution for Java (or Javascript) and ESP8266 to do this already? (Feels like a common problem if implementing "smart home" devices).

Answer 1

There is no single way that we discover devices on a local area network.

Rather than scanning a network, devices generally either use a multicast or broadcast protocol to announce their presence, or they rendezvous at a (usually external) preconfigured server.

Some devices use mDNS - based loosely on the DNS protocol, they multicast packets to advertise their presence on the network. Apple uses mDNS in their products. It's poorly supported on Android, and requires additional software on Windows. mDNS names are normally in the .local domain.

Some devices use UPNP and SSDP - Belkin's Wemo line of products does this. UPNP and SSDP are overly complicated protocols based on XML and SOAP, which are poor choices for devices with limited RAM and processing power like the ESP8266 and ESP32.

Some devices just roll their own protocol. Haiku's "Big Ass Fan" line does this - they broadcast UDP packets with a homegrown protocol that at least initially was vulnerable to a variety of problems. I don't recommend going this route unless you really know what you're doing. Other, established protocols have already had a chance to have the bugs ironed out. Unless you're experienced in protocol design you're more likely to reinvent problems other protocols have had than a wonderful shiny new discoverability protocol.

These kinds of devices will either periodically broadcast or multicast a packet announcing themselves, or the thing you're calling a "hub" would broadcast or multicast a request and the devices would respond to that request.

Not all devices present an interface for direct control over the LAN they're connected to. Some simply rendezvous with a remote server. In this case you discover them by asking the server to enumerate them, and you control them via that server. Google's Nest products work this way - initial provisioning is done via Bluetooth; after that apps communicate with the devices through remote servers.

Devices rendezvousing in this manner are generally preconfigured with the name of the rendezvous server, but they might also be configured with the name of the server during network provisioning (generally an app communicates with them to share wifi credentials; it might also share information about a rendezvous server as well).

We generally do not scan for names in IP address blocks or actively probe for new devices unless we're debugging a network or doing some kind of security sweep.

The process of scanning an IP address block that you described is problematic and unreliable. The only reason it works is that some routers pick up a device's name from the device's DHCP request (or the router may be configured to know the device's name). The router also handles DNS for the devices on the network, generally by forwarding them to the ISP's DNS servers or to DNS servers that the network's owner configured it to use. It intercepts DNS requests for devices whose names it knows and replies to them itself, instead of forwarding them to an external DNS server.

You also have to know about the network configuration to do this properly. Is the network only a /24? What if it's /22? or /16? Are you prepared to scan 2^24 IP addresses if the network is configured as a /8?

Though the router may intercept requests for names and return addresses, it won't necessarily intercept names for addresses and return names.

Scanning also generates an unnecessary network traffic. While your one "hub" scanning may not seem like much, if you have multiple scanners from different makers running it doesn't scale well.

If your "hub" bypasses the router for DNS requests then it also will not be able to resolve names provided by the router.

Also not all routers do this. It's not part of the Internet architecture, it's a convenience feature that some routers provide. You cannot depend on it working.

You could also attempt an active scan of the network, trying to communicate with each IP address on it. We do this for network debugging, but running this continually to detect new devices would be a hostile way to interact with the network.

Network infrastructure devices - switches and routers - of course maintain lists of all active devices on the network. You can access these lists using SNMP (Simple Network Management Protocol), but few consumer switches and routers support SNMP. And allowing access to your network infrastructure to a random piece of software is a network security nightmare.

Your best bet is a simple multicast/broadcast protocol like mDNS. Don't scan, announce or request. It's responsive, doesn't place a burden on the network, doesn't depend on peculiarities of routers, and won't make network administrators hate you.

Answer 2

Long time ago I implemented a similar solution to discover clients connecting to the network. My strategy consisted in taking advantage of the DHCP configuration.

If your devices must ask IP addresses from a Linux/Unix DHCP server under your control, you could configure it to let you know when a device connects or disconnects from the network.

We were using Linux DHCP server, and in the dhcpd.conf there is a section about events. Basically it says that when the DHCP commits to a particular IP address lease for a client it raises an event, and when that happens, it can run a listener you could have defined for that event. In the listener we can ask the DHCP to execute a command, e.g. write the client information (e.g. MAC address) and its IP address lease to a named pipe, and your Java application could just be reading from this queue. (See also dhcp-eval for further details). A similar event can be run when a client releases an IP lease or when a lease expires.

Here's an example

subnet 192.168.1.0 netmask 255.255.255.0 {
    option routers  192.168.1.2;

    on commit {
        set clip = binary-to-ascii(10, 8, ".", leased-address);
        set clhw = binary-to-ascii(16, 8, ":", substring(hardware, 1, 6));
        execute("/usr/local/sbin/dhcpevent", "commit", clip, clhw, host-decl-name);
    }
    ...

If you can control your DHCP config there is so much you can extract from it.