Camera object detection in JavaScript

Question

Suppose the user has a modern browser like Chrome and enables necessary HTML5 camera settings (so that getUserMedia works), how would one detect specific predefined objects shown in webcam sight, using JavaScript?

For instance, there's HTML5/ JS-based face detection which works great, and I saw another hand detection demo (which didn't work well here; I might be doing something wrong). What are the necessary steps to train the camera to detect given other objects of (developer) choice? Say, I want the cam to recognize the location of a red pen; or perhaps the darkest object in sight; or perhaps a black iPhone waved into the camera etc.

Thanks!

Answer 1

Object detection in itself is a very tricky business. You have to know what your object is, whether it is smooth, flexible, has a lot of color contrast, moves quickly, and a lot of other questions before you can determine the best method.

Also, it depends on whether you only want to detect an object, or if you want to track it during its movement in front of the camera.

I'll be only naming a few methods here, because I don't have time to elaborate a lot. You can probably find a lot of documentation on Google once you know the names, but be aware that you might need some mathematical skills if you have to implement them yourself. So, this usually involves :

• Computing descriptors at interesting points. Look at the SIFT or HoG (Histograms of Gradients) descriptors on Google, these are the most used ones.
• Building some kind a recognition structure, which, again, can change greatly, depending upon your object and your descriptors. Popular methods include Neural networks, Support Vector Machines. For moving objects, you can usually add graph-related techniques such as Graph Cuts into the mix.

Again, depending on the object, these might not even be close to the right method.

As far as I know, there's very little software available for all that in JavaScript, but I'd be happy to know it if you do find something. Again, here are a few pointers :

• Your Face Detection sample is using something very popular called a Cascade Classifier, which is available in the even more popular library OpenCV, and is considered by most to be the method of choice for face detection.
• If you can consider moving part of the processing to a server, you could use OpenCV which has tons of available algorithms.

I hope I was able to help you start a little bit ;)