Tensorflow Object detection with multiple camera

Question

I tried GitHub code Object-Detector-App

This works fine for single camera with 1 second latency, but when I tried it for multiple cameras ,(two,three....) for that I create multiple thread for graph and session for each cameras and I got high latency depends upon number of camerase. a) I am using NVIDIA Quadpro GP100 and the camera inputs are HD(1920x1080) b) I am using SSD_VI_COCO_11_06_2017

I studied thread and queue concept in tensorflow, i googled a lot but couldn't find the practical approach for it.

I want to know that am I doing the right way to handle more than one camera for object detection or is there any better approach for that?

Answer 1

First make sure you are reading frames from multiple cameras without latency. If using opencv with the python wrapper be aware the underlying implementation of read() works differently for usb webcams vs ip cameras. Usb cameras can be stacked easily, while ip cameras may require python threading and queues to read frames efficiently.

Unless you need images that large, resize them before feeding them into the feed dict.

30+ FPS should be easily achievable with that gpu. Common issues are loading the graph and starting a new session for every inference. There is no need to open a new graph/session for each camera if using one gpu. Open one graph and one session outside the loop and inference the images from all the cameras in one loop. This can be done one after another or in batches. E.g. stack four images of shape (1, 512, 512, 3) into a single array of shape (4, 512, 512, 3) and the inference for all four images will happen in parallel provided the gpu can fit them into memory.

It might look like this:

Load graph and open session

Start thread for each camera that reads in frames and fills a queue

Start a single inference thread that reads in frames from the queues and either inferences the frames sequentially or in batches