Minimal Example for dlib SVM using Python/Numpy

Question

I need to deploy an SVM in a C++ build target system. Therefore I want to train an SVM using dlib with python/numpy, serialize it and evaluate in the target system.

The python documentation for dlib is rather obscure to me, so can anyone help me with this minimal example?

import dlib

# My data in numpy
feature_column_1 = np.array([-1, -2, -3, 1, 2, 3])
feature_column_2 = np.array([1, 2, 3, -1, -2, -3])
labels = np.array([True, True, True, False, False, False])

# Features
feature_vectors = dlib.vectors()
for feature_column in [feature_column_1, feature_column_2]:
    feature_vectors.append(dlib.vector(feature_column.tolist()))

# Labels
labels_array = dlib.array(labels.tolist())

# Train
svm = dlib.svm_c_trainer_linear()
svm.train(feature_vectors, labels_array)

# Test
y_probibilities = svm.predict(labels_array_new)

I get following error for the training:

---> 18 svm.train(vectors, array)

ValueError: Invalid inputs

Answer 1

I just added an official example for this to dlib. I was surprised to find it wasn't included when I looked. It's available here: https://github.com/davisking/dlib/blob/master/python_examples/svm_binary_classifier.py. Here are the relevant details:

import dlib
import pickle    

x = dlib.vectors()
y = dlib.array()

# Make a training dataset.  Here we have just two training examples.  Normally
# you would use a much larger training dataset, but for the purpose of example
# this is plenty.  For binary classification, the y labels should all be either +1 or -1.
x.append(dlib.vector([1, 2, 3, -1, -2, -3]))
y.append(+1)

x.append(dlib.vector([-1, -2, -3, 1, 2, 3]))
y.append(-1)


# Now make a training object.  This object is responsible for turning a
# training dataset into a prediction model.  This one here is a SVM trainer
# that uses a linear kernel.  If you wanted to use a RBF kernel or histogram
# intersection kernel you could change it to one of these lines:
#  svm = dlib.svm_c_trainer_histogram_intersection()
#  svm = dlib.svm_c_trainer_radial_basis()
svm = dlib.svm_c_trainer_linear()
svm.be_verbose()
svm.set_c(10)

# Now train the model.  The return value is the trained model capable of making predictions.
classifier = svm.train(x, y)

# Now run the model on our data and look at the results.
print("prediction for first sample:  {}".format(classifier(x[0])))
print("prediction for second sample: {}".format(classifier(x[1])))


# classifier models can also be pickled in the same was as any other python object.
with open('saved_model.pickle', 'wb') as handle:
    pickle.dump(classifier, handle)

However, if you want to use C++ you should just use C++. Dlib is primarily a C++ library rather than a python library. The whole point of dlib is to provide a nice C++ API for people who want to do machine learning. So you would be much better off just using C++ for training. There are 99 full C++ examples that come with dlib and complete C++ API documentation. For example, here is a relevant example http://dlib.net/svm_c_ex.cpp.html.

I really should emphasise that dlib's C++ API is much more flexible than the python API. Really, the point of dlib is to make machine learning easy in C++, the python API for dlib is an afterthought. In fact, there are a lot of features of dlib that are expressed using things like C++ templates which have no possible correlate in Python (e.g. since python has nothing like C++ templates) and so these features are not exposed to python. So really, if you want to use C++ then use C++. There is no reason to use the Python API if you know how to write C++.

Answer 2

Not a complete answer, but some remarks:

(1)

The error you observe is at least partially due to a check here.

    for (long r = 0; r < x_labels.nr(); ++r)
    {
        if (x_labels(r) != -1 && x_labels(r) != 1)
            return false;

It means: labels = np.array([True, True, True, False, False, False]) is wrong, while labels = np.array([1, 1, 1, -1, -1, -1]) is correct.

(2)

Usually, in most MLlibs, the data-format is (n_samples, n_features) where rows are observations.

When printing out your feature vectors, this looks correct, but your code, after the change of (1) will still throw the same error, except when you treat it as the other way around: so the other interpretation is: you got 2 samples of 6 features each. Using this assumption, y needs to have 2 values. Et voilà... at least it's training!

Code which is not throwing an error (i'm ignoring what it's really doing)

import dlib
import numpy as np

# My data in numpy
feature_column_1 = np.array([-1, -2, -3, 1, 2, 3])
feature_column_2 = np.array([1, 2, 3, -1, -2, -3])
labels = np.array([-1, 1])  # +1/-1 & size == 2

# Features
feature_vectors = dlib.vectors()
for feature_column in [feature_column_1, feature_column_2]:
    feature_vectors.append(dlib.vector(feature_column.tolist()))

# Labels
labels_array = dlib.array(labels.tolist())

# Train
svm = dlib.svm_c_trainer_linear()
svm.be_verbose = 10

svm.train(feature_vectors, labels_array)
print('k')

(3)

dlib's python-API does not have an predict() function. The C++-API has some learned_function attribute used for prediction, but this and maybe this indicates you have to do that by yourself (probably using c_class1 and co. as i can't map anything else to something usable from the candidates of the API-doc).

EDIT: I was wrong about predict, as dlib's maintainer/developer mentioned in the comments!

I really really don't like the state of this and would use something else! (or improve dlib).

Not sure how to interpret your environment-constraints but:

• libsvm / liblinear are still state-of-the-art for tasks like these and they are build to be used from C++!
• and sklearn is by far the nicest way to do this from python (internally using the libraries above)