How to use IP address as a feature in a neural network

Question

Using Keras, I want to build an LSTM neural net to analyze user behavior in my system. One of my features is a string containing the user IP address, that could be IPv4 or IPv6.

As I see it I need to embed the address so it can be use as a feature. In Keras documentation there is no clear explanation how to do such a thing.

What would be a good place to start?

Answer 1

The optimal way to encode IP addresses in your model depends on their semantics with respect to your problem. There are several options:

One-hot encoding

This way assumes no relationship between IP addresses at all. 1.2.3.4 is assumed to be as different from 1.2.3.5 as 255.255.255.255. To prevent having 2^32 features, you only encode the IP addresses in your training data as features and treat new IP's as unkown. One way to achieve this is sklearn's LabelBinarizer:

train_data = ['127.0.0.1', '8.8.8.8', '231.58.91.112', '127.0.0.1']
test_data = ['8.8.8.8', '0.0.0.0']

ip_encoder = LabelBinarizer()
print('Train Inputs:\n', ip_encoder.fit_transform(train_data))
print('Test Inputs:\n', ip_encoder.transform(test_data))

This prints:

Train Inputs:
 [[1 0 0]
 [0 0 1]
 [0 1 0]
 [1 0 0]]
Test Inputs:
 [[0 0 1]
 [0 0 0]]

Note the difference between One-hot encoding and dummy encoding.

Using 32 or 128 features

Here, you use one feature per bit in the IP.

Advantages:

1. The model can more easily identify IP's that belong to the same subnet.
2. The number of features remains small even for a large number of distinct IP addresses in your training data.

Disadvantages:

1. The model doesn't know how subnets work. If your training data actually justifies generalizing multiple IP's to their subnet, there is a high probability that the model won't apply the subnet mechanism 100% correctly. What I mean is that it might learn to use the 2nd and 3rd part of 1.1.1.1 and 1.1.1.2 to detect this specific subnet and thus treat 0.1.1.1 as an IP of this subnet as well.
2. Reducing the number of features is great but it also makes it harder for the model to detect whether two IP addresses are the same. When using One-Hot-Encoding it has this information directly in the features, while with this approach it would need to learn 32 / 128 'if' statements internally to see whether an IP address is the same. But a neural network is unlikely to learn this completely if fewer 'if' statements suffice to discriminate correctly. This is analogous to the treatment of subnets. For example, if '1.2.3.4' is a very discriminative IP in your training data, i.e. this IP makes a specific outcome very likely, the model will probably learn to detect this IP based on a specific subset of its bits. Thus, different IPs with the same value for these specific bits will be treated similarly by the model.

Overall, this approach needs to be treated carefully.

One-hot encoding frequent IPs

If the number of distinct IPs is too high to create a new feature for each IP, you can check if each IP is actually important enough to be incorporated into the model. For example, you might check the histogram of IPs. IPs that only have a few samples in the training data might be worth ignoring. With only a few samples, the model is likely to either overfit on these IPs or ignore them completely. So, you could one-hot-encode the top 1000 frequent IPs in your training data and add one feature for all other IPs. Similarly, you could try to do some data preprocessing and cluster the IPs based on their location etc.

Using numerical inputs

It might be tempting to use a single int32 feature or four int8 features for an IPv4. This is a bad idea as it allows the model to do arithmetics on IPs, such as 1.1.1.1 + 2.2.2.2 = 3.3.3.3.

Word Embeddings

This is the way that you linked to in the question (https://keras.io/layers/embeddings/). These embeddings are intended for Word Embeddings and should be trained on sentences / text. They generally shouldn't be used for encoding IPs.