Dlib face detection terrible performance on C++, good in python, why?

Question

I am trying to write a simple face detection algorithm using OpenCV for camera capture and Dlib for face detection (using Histogram of Oriented Gradients algorithm).

Using Python, I get a decent performance with around 20 fps. However, the same-ish code in C++ has very poor performance, with each dlib's detection process taking around 4 seconds.

Does anyone know what's happening ?

I did some optimization, but nothing really improve performances :

• image is reduced to 640x480
• I compiled dlib with AVX instructions enabled
• I also tried to compile with -0fast flag...

Here are the codes:

In C++:

#include "opencv2/objdetect.hpp"
#include "opencv2/highgui.hpp"
#include "opencv2/imgproc.hpp"
#include <dlib/opencv.h>
#include <dlib/image_processing/frontal_face_detector.h>
#include <dlib/image_processing.h>

using namespace dlib;
using namespace std;

int main(){

cv::VideoCapture cap(0);
vector<cv::Rect> facesCV;
vector<rectangle> faces;
frontal_face_detector detector = get_frontal_face_detector();
cv::namedWindow("test");
cv::Mat frame, small;

if (!cap.isOpened()) {
    cerr << "Unable to connect to camera" << endl;
    return 1;
}

while (true) {
    // Grab a frame
    if (!cap.read(frame)) {
        break;
    }
    cv::resize(frame, small, {640, 480});
    cv_image<rgb_pixel> cimg(small);

    // Detect faces
    faces = detector(cimg);
    for (auto &f : faces) {
        facesCV.emplace_back(cv::Point((int) f.left(), (int) f.top()), cv::Point((int) f.right(), (int) f.bottom()));
    }

    for (auto &r : facesCV) {
        cv::rectangle(small, r, {0, 255, 0}, 2);
    }
    cv::imshow("test", small);
    cv::waitKey(1);
    faces.clear();
    facesCV.clear();
}
}

In Python :

import argparse
import cv2
import dlib

#initialize face detector
detector = dlib.get_frontal_face_detector()

#initialize video source
cam = cv2.VideoCapture(0)
window = cv2.namedWindow("camera")

while True:
    ret, image = cam.read()
    if ret is True:
        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        gray =cv2.resize(gray, (640, 480))

        for r in detector(gray, 0):
            cv2.rectangle(image, (r.left(), r.top()), (r.right(), r.bottom()), (0, 255, 0), 2)

        cv2.imshow(window, image)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

    else:
        break

cam.release()
cv2.destroyAllWindows()

Answer 1

The issue came from the CMakeLists.txt. AVX optimizations need to be set in the CMakeLists.txt this way :

set(USE_AVX_INSTRUCTIONS ON CACHE BOOL "Use AVX instructions")
add_subdirectory("path/to/dlib" dlib_build)

add_executable(myProject main.cpp)
target_link_libraries( myProject dlib::dlib)

Answer 2

The accepted solution wasn't a solution for me.

I was building dlib separately (using the option: -DUSE_AVX_INSTRUCTIONS=ON) and then attempting to build my project with this in my CMakeLists.txt file:

find_package(dlib REQUIRED)

It sort of worked. It was linking to dlib, but for some reason it ran super slow.

To get the most out of dlib I had to:

add_subdirectory(../dlib dlib_build)

in my CMakeLists.txt file and build my project as I would have built dlib:

cmake -DUSE_AVX_INSTRUCTIONS=ON  ../
cmake --build . --config Release